-- Logs begin at Wed 2022-09-21 15:42:00 UTC, end at Wed 2022-09-21 17:13:45 UTC. -- Sep 21 16:06:51.346190 dv-cinder-858621-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 21 16:06:51.355172 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 21 16:06:51.357213 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 21 16:06:51 2022] *** Sep 21 16:06:51.357213 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 21 16:06:51.357213 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 21 16:06:51.357213 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: nodename: dv-cinder-858621-1-iscsi Sep 21 16:06:51.357213 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: machine: x86_64 Sep 21 16:06:51.357213 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: clock source: unix Sep 21 16:06:51.357213 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: pcre jit disabled Sep 21 16:06:51.357213 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: detected number of CPU cores: 4 Sep 21 16:06:51.357213 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: current working directory: / Sep 21 16:06:51.357213 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: detected binary path: /usr/bin/uwsgi-core Sep 21 16:06:51.357882 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: your processes number limit is 47680 Sep 21 16:06:51.357882 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: your memory page size is 4096 bytes Sep 21 16:06:51.357882 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: detected max file descriptor number: 2048 Sep 21 16:06:51.357882 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: lock engine: pthread robust mutexes Sep 21 16:06:51.357882 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: thunder lock: enabled Sep 21 16:06:51.357882 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 21 16:06:51.357882 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 21 16:06:51.397090 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: Python main interpreter initialized at 0x55be51fba9c0 Sep 21 16:06:51.397090 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: python threads support enabled Sep 21 16:06:51.397090 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: your server socket listen backlog is limited to 100 connections Sep 21 16:06:51.397090 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: your mercy for graceful operations on workers is 90 seconds Sep 21 16:06:51.397468 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: mapped 403077 bytes (393 KB) for 2 cores Sep 21 16:06:51.397547 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: *** Operational MODE: preforking *** Sep 21 16:06:51.397789 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: *** uWSGI is running in multiple interpreter mode *** Sep 21 16:06:51.397789 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: spawned uWSGI master process (pid: 81414) Sep 21 16:06:51.397989 dv-cinder-858621-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 21 16:06:51.399844 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: spawned uWSGI worker 1 (pid: 81425, cores: 1) Sep 21 16:06:51.399844 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: spawned uWSGI worker 2 (pid: 81426, cores: 1) Sep 21 16:06:51.399844 dv-cinder-858621-1-iscsi devstack@keystone.service[81414]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 21 16:06:55.720290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 21 16:06:55.720290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: from cryptography.utils import int_from_bytes Sep 21 16:06:55.721081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 21 16:06:55.721081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: from cryptography.utils import int_from_bytes Sep 21 16:06:55.721365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 21 16:06:55.721365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: from cryptography.utils import int_from_bytes Sep 21 16:06:55.721815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 21 16:06:55.721815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: from cryptography.utils import int_from_bytes Sep 21 16:06:55.771033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 21 16:06:55.771199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 21 16:06:55.771330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] command line args: [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 21 16:06:55.771597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 21 16:06:55.771774 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ================================================================================ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 21 16:06:55.771969 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 21 16:06:55.772061 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] admin_token = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.772168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 21 16:06:55.772254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] config_dir = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.772364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] command line args: [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 21 16:06:55.772442 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.772597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] config_source = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.772678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 21 16:06:55.772775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] debug = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.772858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ================================================================================ {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 21 16:06:55.772972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.773231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] admin_token = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.773294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] config_dir = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.773349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.773473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.773573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.773667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] config_source = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.773780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.773882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] debug = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.773979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.774119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.774266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.774420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.774543 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_config_append = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.774644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.774740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.774840 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.774935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_dir = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.774935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_file = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.775121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.775216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.775317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.775317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_config_append = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.775497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.775613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.775725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.775803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_dir = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.775803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_file = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.775983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.775983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.776170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.776284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.776364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.776483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.776562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.776683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.776763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.776835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.776907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.777010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.777100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.777238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.777350 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.777491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.777600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.777713 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.777853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.777990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.778135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.778300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.778455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.778568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.778701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.778818 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.778927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.779067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] publish_errors = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.779067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.779276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.779410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.779520 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.779629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.779736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.779849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] publish_errors = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.779953 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.780028 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.780162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] standard_threads = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.780262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.780368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.780443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.780610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.780696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.780775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.780845 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.780945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_journal = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.781062 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] standard_threads = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.781140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_json = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.781277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_stderr = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.781408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.781482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_syslog = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.781550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.781662 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.781737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.781854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] use_journal = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.781941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.782032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] use_json = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.782165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.782251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] use_stderr = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.782392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] use_syslog = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.782473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.782623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 21 16:06:55.782706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.782880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.782962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.783114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.783188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.783363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.783440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.783597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.783673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.783845 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.783921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.784109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.784192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.784359 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.784435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.784596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.784677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.784840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.784920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.785073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.785154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.785320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.785399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.785552 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.785631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.785796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.785871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.786029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.786134 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.786301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.786378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.786547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.786628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.786782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.786905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.787059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.787136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.787278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.787353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.787523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.787610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.787760 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.787836 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.788018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.788108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.788219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.788302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.788448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.788524 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.788679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.788755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.788925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.789008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.789173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.789254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.789401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.789476 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.789629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.789712 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.789877 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.789951 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.790146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.790236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.790380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.790526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.790608 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.790677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.790845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.790925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.791078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.791161 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.791311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.791386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.791542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.791618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.791772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.791860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.792002 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.792077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.792231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.792306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.792461 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.792606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.792677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.792745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.792910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.792986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.793141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.793225 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.793367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.793447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.793599 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.793680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.793831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.793915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.794081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.794168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.794318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.794394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.794551 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.794631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.794783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.794848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.795015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.795079 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.795249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.795323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.795482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.795610 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.795753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.795897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.796015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.796149 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.796256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.796362 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.796494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.796605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.796724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.796860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.796969 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.797085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.797228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.797335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.797479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.797589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.797707 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.797851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.797851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.798151 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.798151 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.798356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.798505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.798613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.798724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.798866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.798973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.799080 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.799223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.799331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.799438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.799571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.799571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.799867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.799987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.800096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.800096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.800272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.800343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.800439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.800531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.800605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.800674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.800743 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.800844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.800921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.800999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.801070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.801219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.801301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.801453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.801560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.801686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.801764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.801918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.801994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.802174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.802264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.802412 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.802499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.802642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.802720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.802896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.802980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.803113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.803191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.803354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.803431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.803588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.803665 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.803809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.803887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.804042 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.804119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.804270 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.804356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.804500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.804585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.804730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.804807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.804962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.805038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.805190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.805278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.805422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.805528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.805639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.805717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.805874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.805958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.806126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.806208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.806368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.806446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.806602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.806691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.806851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.806929 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.807082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.external = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.807169 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.807318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.807475 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.807566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.807649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.807813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.807895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.808046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.external = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.808133 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.password = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.808276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.token = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.808413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.808530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.808645 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.808760 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.808873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.808987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.809112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.password = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.809226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.809340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] auth.token = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.809453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.809573 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.809689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.809809 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.809925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.810061 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.810179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.810292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.810406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.810531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.810654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.810767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.810881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.810994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.811114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.811237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.811351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.811464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.811577 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.811692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.811805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.811926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.812040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.812154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.812267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.812380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.812494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.812615 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.812729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.812843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.812957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.813070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.813182 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.813304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.813418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.813532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.813644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.813757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.813871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.813994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.814139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.814255 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.814368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.814481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.814630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.814761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.814878 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.814992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.815105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.815219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.815897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.816013 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.816127 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.816240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.816354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.816468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.816589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.816705 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.816818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.816932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.817052 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.817166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.817288 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.817403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.817516 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.817633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.817755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.817871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.818076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.818195 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.818341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.818457 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.818570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.818694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.818818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.818933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.819047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.819167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.819281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.819401 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.819523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.819646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.819760 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.819874 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.819988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.820102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.820224 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.820338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.820452 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.820565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.820678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.820792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.820913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.821028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.821142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.821246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.821358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.821472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.821594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.821708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.821827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.821949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.822118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.822247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.822371 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.822485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.822599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.822713 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.822826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.822940 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.822940 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.823162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.823162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.823376 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.823490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.823603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.823724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.823839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.823952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.824065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.824178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.824292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.824413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.824527 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.824640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.824753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.824866 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.824979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.825104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.825219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.825333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.825447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.825559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.825680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.825680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.825894 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.826007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.826130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.826244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.826369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.826485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.826598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.826598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.826811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.826924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.827046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.827160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.827273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.827386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.827500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.827616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.827737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.827852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.827965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.828078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.828190 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.828303 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.828458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.828576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.828689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.828802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.828921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.829035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.829156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.829156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.829371 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.829485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.829598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.829598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.830117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.830237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.830350 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.830464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.830579 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.830692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.830814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.831514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.831694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.831882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.832061 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.832247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.832415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.832572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.832740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.832740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.833034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.833183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.833354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.833520 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.833687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.833848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.833967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.834099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.834229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.834229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.834446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.834559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.834672 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.834787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.834908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.834908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.835121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.835235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.835348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.835461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.835582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.835696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.835809 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.835923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.836036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.836149 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.836270 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.836385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.836497 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.836610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.836723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.836836 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.836957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.837071 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.837184 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.837297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.837411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.837523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.837647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.837762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.837875 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.837989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.838139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.838254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.838380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.838527 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.838643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.838756 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.838869 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.838983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.839104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.839219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.839333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.839446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.839559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.839672 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.839793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.839908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.840021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.840134 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.840247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.840360 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.840481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.840595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.840700 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.840802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.840913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.841026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.841148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.841262 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.841375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.841488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.841601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.841788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] role.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.841904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.842018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] role.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.842151 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.842266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.842379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.842505 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.842620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.842733 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.842847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] role.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.842960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.843073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] role.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.843196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.843311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.843424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.843792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.843909 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.844022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.844145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.844261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.844375 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.844488 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.844602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.844715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.844837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.844952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.845065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.845179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.845294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.845415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.845538 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.845659 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.845659 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.845874 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.845874 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.846144 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.846273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.846391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.846506 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.846619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.846733 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.846846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.846968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.847083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.847196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.847310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.847424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.847546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.847665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.847777 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.847891 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.848004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.848117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.848238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.848353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.848468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.848612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.848728 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.848842 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.848964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.849079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.849192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.849305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.849418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.849532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.849653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.849768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.849881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.849994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.849994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.850219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.850345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.850461 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.850576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.850689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.850802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.850915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.850915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.851144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.851249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.851361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.851474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.851588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.851710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.851824 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.851938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.852051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.852163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.852276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.852397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.852511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.852625 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.852739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.852739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.852952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.853074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.853188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.853188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.853401 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.853401 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.853628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.853752 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.853875 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.853875 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.854137 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.854253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.854367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.854367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.854592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.854592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.854817 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.854944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.855059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.855059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.855274 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.855387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.855509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.connection = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.855624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.855738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.855851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.855964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.856078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.856199 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.856314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.856427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.856542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.856655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.856769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.856891 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.857007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.857120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.857234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.857347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.857461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.857586 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.857703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.857816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.857931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.858057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.858173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.858299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.858414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.858528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.858677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 21 16:06:55.858794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.858794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.858794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.858794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.858794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 21 16:06:55.858794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 21 16:06:55.887249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.887481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.887709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.890578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.890948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.891180 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.896023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.899547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.923630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.923856 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.925786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.926019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.977459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:55.978861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.027970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.028143 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.028283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.028520 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.186524 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.186760 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.186853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.186977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.187085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.187216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.187306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.187413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.187495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.187631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.187713 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.187849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.212272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.212506 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.212593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.212737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.212819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.212961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.213042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.213178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.213267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.213394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.213476 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.213609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.213690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.213823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.225697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.225947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.239305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.239467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.239546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.239644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.239763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.239847 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.239990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.240074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.240212 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.240296 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.240428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.240511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.240646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.240739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 21 16:06:56.258839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.259037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 21 16:06:56.259109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.259308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 21 16:06:56.259414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.259582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 21 16:06:56.259665 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.259860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 21 16:06:56.259942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.260121 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.260211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.260384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.260468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.260644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.260728 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.260910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.271381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.271638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.271725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.271936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.272019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.272210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.272292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.272501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.279798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.279975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.282937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.283114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.283246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.283411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.283535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.283704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.283814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.284026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.284131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.284322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.284404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.284594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.284678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.284874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.284957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.285155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.285245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.285437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.285519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.285709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.285791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.285981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.309345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.309575 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.310950 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.311155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.311239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.311450 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.316705 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.316883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.317003 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.317186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.317292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.317471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.323878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.324039 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.324182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.324338 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.330399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.330613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.330698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.330972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.331545 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.331760 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.331844 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.332053 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.332135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.332329 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.332419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.332601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.332683 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.332871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.332951 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.333141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.348740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.348841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.348912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.349205 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.352711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.353094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.355990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.356286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.356414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.356714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.360541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.360990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.363851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.364220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.367129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.367432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.367541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.367719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.367844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.368010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.368131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.368290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.368418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.368569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.368711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.369003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.385054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.385346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.385566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.385650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.385858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.385954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.386166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.386433 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.392755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.393428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.394863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.395511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.396405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.396682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.397060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.397467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.399499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.399793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.400077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.400158 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.400364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.400471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.400696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.400776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.400984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.401383 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.411998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.412278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.412561 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.412715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.412840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.413006 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.413230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.413312 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.413517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.413599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.413798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.413953 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.414275 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.414562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.429277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.430233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.432224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.433171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.435169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.436113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.436678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.437564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.438202 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.438488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.438894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.439185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.439472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.439885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.445869 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.447073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.447373 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.448709 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.450307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.450706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.450994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.451276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.452082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.452556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.452914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.453245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.460348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.460756 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.461127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.462214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.462503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.462787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.469756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.470041 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.470348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.470634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.470911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.470992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 21 16:06:56.471186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.471277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.471484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.471570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.471772 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.471854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.472048 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.472139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.472332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.472413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.472609 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.472690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.472878 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.472958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.473146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.473226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.473409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.473493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.473696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.473968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.474263 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.474543 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 21 16:06:56.508396 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55be51fba9c0 pid: 81426 (default app) Sep 21 16:06:56.509421 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55be51fba9c0 pid: 81425 (default app) Sep 21 16:06:56.511852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8c829-0af8-4601-b068-4a79657d324d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:06:56.512140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8c829-0af8-4601-b068-4a79657d324d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:06:56.512417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8c829-0af8-4601-b068-4a79657d324d None None] PATH_INFO: `/v3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:06:56.516607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1/1] 10.222.0.64 () {50 vars in 791 bytes} [Wed Sep 21 16:06:56 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:03.450240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd080f-55a4-4619-b9b1-98dec7411bbb None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:03.450852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd080f-55a4-4619-b9b1-98dec7411bbb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:03.450852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd080f-55a4-4619-b9b1-98dec7411bbb None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:03.455370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1/2] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:07:03.460872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e6c58-0959-4f2b-a27c-43c967404edc None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:03.461395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e6c58-0959-4f2b-a27c-43c967404edc None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:03.461515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e6c58-0959-4f2b-a27c-43c967404edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:03.531155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: INFO dbcounter [None req-e98e6c58-0959-4f2b-a27c-43c967404edc None None] Registered counter for database keystone Sep 21 16:07:03.550697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG oslo_db.sqlalchemy.engines [None req-e98e6c58-0959-4f2b-a27c-43c967404edc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81426) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 21 16:07:03.556870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writer thread running {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 21 16:07:03.737282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG passlib.handlers.bcrypt [None req-e98e6c58-0959-4f2b-a27c-43c967404edc None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81426) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 21 16:07:03.737584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG passlib.handlers.bcrypt [None req-e98e6c58-0959-4f2b-a27c-43c967404edc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81426) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 21 16:07:03.795762 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e98e6c58-0959-4f2b-a27c-43c967404edc None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:03.797382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e98e6c58-0959-4f2b-a27c-43c967404edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:03.927352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 2/3] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 21 16:07:04.423706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd080f-55a4-4619-b9b1-98dec7411bbb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:04.493105 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: INFO dbcounter [None req-a2dd080f-55a4-4619-b9b1-98dec7411bbb None None] Registered counter for database keystone Sep 21 16:07:04.515098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG oslo_db.sqlalchemy.engines [None req-a2dd080f-55a4-4619-b9b1-98dec7411bbb None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81425) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 21 16:07:04.521994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writer thread running {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 21 16:07:04.641235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd080f-55a4-4619-b9b1-98dec7411bbb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:04.646154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d642bd-a62c-4ec3-98d5-796f41a1d289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:04.646789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d642bd-a62c-4ec3-98d5-796f41a1d289 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:04.647077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d642bd-a62c-4ec3-98d5-796f41a1d289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:04.647351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d642bd-a62c-4ec3-98d5-796f41a1d289 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:04.655604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d642bd-a62c-4ec3-98d5-796f41a1d289 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:04.711120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: WARNING keystone.common.rbac_enforcer.enforcer [None req-40d642bd-a62c-4ec3-98d5-796f41a1d289 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 21 16:07:04.711395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d642bd-a62c-4ec3-98d5-796f41a1d289 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:04.721707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 2/4] 10.222.0.64 () {58 vars in 1324 bytes} [Wed Sep 21 16:07:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 16:07:04.727725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e6c58-0959-4f2b-a27c-43c967404edc None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:04.744532 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e6c58-0959-4f2b-a27c-43c967404edc None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:04.748859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17832f1-51a5-4940-9bff-14e45d169437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:04.749971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17832f1-51a5-4940-9bff-14e45d169437 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:04.750277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17832f1-51a5-4940-9bff-14e45d169437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:04.750559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17832f1-51a5-4940-9bff-14e45d169437 admin admin] PATH_INFO: `/v3/projects/92877a675fac490b8473d8157aef4217` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:04.752708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17832f1-51a5-4940-9bff-14e45d169437 admin admin] RBAC: Authorizing `identity:get_project(project_id=92877a675fac490b8473d8157aef4217)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:04.805641 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a17832f1-51a5-4940-9bff-14e45d169437 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 21 16:07:04.805982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17832f1-51a5-4940-9bff-14e45d169437 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:04.810580 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 3/5] 10.222.0.64 () {56 vars in 1207 bytes} [Wed Sep 21 16:07:04 2022] GET /identity/v3/projects/92877a675fac490b8473d8157aef4217 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:06.942712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9439f-2d4c-4909-a469-ae30fbf3b0f7 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:06.943249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9439f-2d4c-4909-a469-ae30fbf3b0f7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:06.943249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c9439f-2d4c-4909-a469-ae30fbf3b0f7 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:06.947375 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 3/6] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:07:06.952853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602a86c-8443-4c81-8486-e0d6d1c36745 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:06.953285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602a86c-8443-4c81-8486-e0d6d1c36745 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:06.953404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a602a86c-8443-4c81-8486-e0d6d1c36745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:06.974202 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a602a86c-8443-4c81-8486-e0d6d1c36745 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:06.975713 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a602a86c-8443-4c81-8486-e0d6d1c36745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:06.984331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 4/7] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 21 16:07:07.247903 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9439f-2d4c-4909-a469-ae30fbf3b0f7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:07.259983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c9439f-2d4c-4909-a469-ae30fbf3b0f7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:07.264212 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24001c6-a38f-4f01-adb3-e4f50bd23836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:07.264809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24001c6-a38f-4f01-adb3-e4f50bd23836 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:07.265094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24001c6-a38f-4f01-adb3-e4f50bd23836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:07.265371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24001c6-a38f-4f01-adb3-e4f50bd23836 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:07.273509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24001c6-a38f-4f01-adb3-e4f50bd23836 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:07.275295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24001c6-a38f-4f01-adb3-e4f50bd23836 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:07.284607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 4/8] 10.222.0.64 () {58 vars in 1324 bytes} [Wed Sep 21 16:07:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 16:07:07.290257 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602a86c-8443-4c81-8486-e0d6d1c36745 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:07.299683 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a602a86c-8443-4c81-8486-e0d6d1c36745 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:07.304105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a3b1f6-5e82-4974-a131-77db898c57e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:07.304737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3b1f6-5e82-4974-a131-77db898c57e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:07.305020 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3b1f6-5e82-4974-a131-77db898c57e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:07.305296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a3b1f6-5e82-4974-a131-77db898c57e6 admin admin] PATH_INFO: `/v3/users/b9a31b5d481f471a90714fa648bfed35` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:07.306649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3b1f6-5e82-4974-a131-77db898c57e6 admin admin] RBAC: Authorizing `identity:get_user(user_id=b9a31b5d481f471a90714fa648bfed35)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:07.308475 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a3b1f6-5e82-4974-a131-77db898c57e6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:07.310008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 5/9] 10.222.0.64 () {56 vars in 1198 bytes} [Wed Sep 21 16:07:07 2022] GET /identity/v3/users/b9a31b5d481f471a90714fa648bfed35 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:09.582411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c002ac-6f5b-4a0d-869b-7d9e819bd428 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:09.582990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c002ac-6f5b-4a0d-869b-7d9e819bd428 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:09.582990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c002ac-6f5b-4a0d-869b-7d9e819bd428 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:09.584293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 5/10] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:09.590107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dec67-2532-405a-8086-19aa4ca73906 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:09.590598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dec67-2532-405a-8086-19aa4ca73906 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:09.590699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656dec67-2532-405a-8086-19aa4ca73906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:09.596970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607eacf-e040-4108-b9d9-d0570fda44ab None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:09.597257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607eacf-e040-4108-b9d9-d0570fda44ab None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:09.597536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607eacf-e040-4108-b9d9-d0570fda44ab None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:09.598964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 6/11] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:07:09.604314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012cde1-4d51-4511-a9fc-ef8c6d6ce930 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:09.604601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012cde1-4d51-4511-a9fc-ef8c6d6ce930 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:09.604890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012cde1-4d51-4511-a9fc-ef8c6d6ce930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:09.609351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-656dec67-2532-405a-8086-19aa4ca73906 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:09.610559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-656dec67-2532-405a-8086-19aa4ca73906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:09.647276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG passlib.handlers.bcrypt [None req-5012cde1-4d51-4511-a9fc-ef8c6d6ce930 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81425) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 21 16:07:09.647763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG passlib.handlers.bcrypt [None req-5012cde1-4d51-4511-a9fc-ef8c6d6ce930 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81425) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 21 16:07:09.692725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 6/12] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:09.698499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5012cde1-4d51-4511-a9fc-ef8c6d6ce930 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:09.700113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5012cde1-4d51-4511-a9fc-ef8c6d6ce930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:09.803360 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 7/13] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:07:09.960096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656dec67-2532-405a-8086-19aa4ca73906 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:09.987278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:10.071736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656dec67-2532-405a-8086-19aa4ca73906 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:10.074410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012cde1-4d51-4511-a9fc-ef8c6d6ce930 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:10.107474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143abcc-6265-4106-86e4-191f75817bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:10.108125 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143abcc-6265-4106-86e4-191f75817bad None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:10.108410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143abcc-6265-4106-86e4-191f75817bad None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:10.108692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143abcc-6265-4106-86e4-191f75817bad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:10.120810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143abcc-6265-4106-86e4-191f75817bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-7143abcc-6265-4106-86e4-191f75817bad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:10.129778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:10.131002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:07:10.131824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 7/14] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:07:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:10.136695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143abcc-6265-4106-86e4-191f75817bad None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:10.181088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012cde1-4d51-4511-a9fc-ef8c6d6ce930 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:10.183572 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:10.217976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:10.219854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea07e44-60e9-4965-b330-3f1c94d2a179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:10.221369 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea07e44-60e9-4965-b330-3f1c94d2a179 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:10.221651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea07e44-60e9-4965-b330-3f1c94d2a179 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:10.221927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea07e44-60e9-4965-b330-3f1c94d2a179 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:10.237840 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143abcc-6265-4106-86e4-191f75817bad None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:10.280220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61108cb-1635-42c3-b158-fda9a7fd6fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:10.280832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61108cb-1635-42c3-b158-fda9a7fd6fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:10.281113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61108cb-1635-42c3-b158-fda9a7fd6fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:10.281390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61108cb-1635-42c3-b158-fda9a7fd6fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:10.282095 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61108cb-1635-42c3-b158-fda9a7fd6fd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:10.282408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61108cb-1635-42c3-b158-fda9a7fd6fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:10.284347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61108cb-1635-42c3-b158-fda9a7fd6fd4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:10.290949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea07e44-60e9-4965-b330-3f1c94d2a179 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:10.292462 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea07e44-60e9-4965-b330-3f1c94d2a179 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:10.296183 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 8/15] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:10.301986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61108cb-1635-42c3-b158-fda9a7fd6fd4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:10.395671 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 8/16] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 16:07:10.401189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea07e44-60e9-4965-b330-3f1c94d2a179 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:10.403870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61108cb-1635-42c3-b158-fda9a7fd6fd4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:10.404783 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:10.439152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07ce88-65d1-4149-b171-4206da66a7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:10.439745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07ce88-65d1-4149-b171-4206da66a7af None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:10.440024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07ce88-65d1-4149-b171-4206da66a7af None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:10.440297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07ce88-65d1-4149-b171-4206da66a7af None admin] PATH_INFO: `/v3/users/b9a31b5d481f471a90714fa648bfed35` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:10.441488 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07ce88-65d1-4149-b171-4206da66a7af None admin] RBAC: Authorizing `identity:get_user(user_id=b9a31b5d481f471a90714fa648bfed35)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:10.442972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07ce88-65d1-4149-b171-4206da66a7af None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:10.444746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 9/17] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:10 2022] GET /identity/v3/users/b9a31b5d481f471a90714fa648bfed35 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:10.450094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07ce88-65d1-4149-b171-4206da66a7af None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:10.465443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:10.509745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea07e44-60e9-4965-b330-3f1c94d2a179 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:10.543874 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6fcff-7cdc-4bed-92c7-b2f78c7aa663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:10.544724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6fcff-7cdc-4bed-92c7-b2f78c7aa663 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:10.545009 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6fcff-7cdc-4bed-92c7-b2f78c7aa663 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:10.545288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e6fcff-7cdc-4bed-92c7-b2f78c7aa663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:10.555287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07ce88-65d1-4149-b171-4206da66a7af None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:10.565361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6fcff-7cdc-4bed-92c7-b2f78c7aa663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:10.566894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e6fcff-7cdc-4bed-92c7-b2f78c7aa663 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a8e6fcff-7cdc-4bed-92c7-b2f78c7aa663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:07:10.589497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:10.590926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:07:10.591720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:07:10.591720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:07:10.591720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:10.591720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 9/18] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:07:10.591938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1f1be-1004-49f7-b6ac-7f75de86652c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:10.591938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1f1be-1004-49f7-b6ac-7f75de86652c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:10.591938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1f1be-1004-49f7-b6ac-7f75de86652c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:10.592391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b1f1be-1004-49f7-b6ac-7f75de86652c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:10.593314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1f1be-1004-49f7-b6ac-7f75de86652c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:10.594779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b1f1be-1004-49f7-b6ac-7f75de86652c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:10.596310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 10/19] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:10.596531 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6fcff-7cdc-4bed-92c7-b2f78c7aa663 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:10.601846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1f1be-1004-49f7-b6ac-7f75de86652c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:10.652968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:10.681385 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:10.699639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e6fcff-7cdc-4bed-92c7-b2f78c7aa663 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:10.701178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b1f1be-1004-49f7-b6ac-7f75de86652c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:10.734591 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe09b53-a04a-4a2b-94f7-2509c71bd53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:10.735209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09b53-a04a-4a2b-94f7-2509c71bd53b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:10.735496 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09b53-a04a-4a2b-94f7-2509c71bd53b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:10.735776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe09b53-a04a-4a2b-94f7-2509c71bd53b None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:10.736081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0febd3b-5d32-4b86-b207-58fe9326b5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:10.736433 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe09b53-a04a-4a2b-94f7-2509c71bd53b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:10.736695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0febd3b-5d32-4b86-b207-58fe9326b5ba None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:10.736791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe09b53-a04a-4a2b-94f7-2509c71bd53b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:10.737000 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0febd3b-5d32-4b86-b207-58fe9326b5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:10.737271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0febd3b-5d32-4b86-b207-58fe9326b5ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:10.737998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0febd3b-5d32-4b86-b207-58fe9326b5ba None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.domain.id=default, user.id=b9a31b5d481f471a90714fa648bfed35) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:10.738143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe09b53-a04a-4a2b-94f7-2509c71bd53b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:10.738335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0febd3b-5d32-4b86-b207-58fe9326b5ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:10.739882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0febd3b-5d32-4b86-b207-58fe9326b5ba None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:10.746690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 11/20] 10.222.0.64 () {56 vars in 1322 bytes} [Wed Sep 21 16:07:10 2022] GET /identity/v3/role_assignments?user.id=b9a31b5d481f471a90714fa648bfed35&scope.domain.id=default&role.id=459d01a218314543a7a8329e88b78331 => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:10.757087 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 10/21] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:13.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79731cd0-8690-497c-9694-8755eed764ef None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:13.242010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79731cd0-8690-497c-9694-8755eed764ef None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:13.242010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79731cd0-8690-497c-9694-8755eed764ef None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:13.243432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 12/22] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:13.250357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de39557-a15d-47ae-bc28-4cab19e235ba None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:13.250893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de39557-a15d-47ae-bc28-4cab19e235ba None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:13.251336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de39557-a15d-47ae-bc28-4cab19e235ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:13.294179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6de39557-a15d-47ae-bc28-4cab19e235ba None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:13.296208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6de39557-a15d-47ae-bc28-4cab19e235ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:13.437817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b83a4-28de-4d9c-afad-4d3fcc33b7c2 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:13.439873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b83a4-28de-4d9c-afad-4d3fcc33b7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:13.441006 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289b83a4-28de-4d9c-afad-4d3fcc33b7c2 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:13.445472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 13/23] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:13.448966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 11/24] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:13.456261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880fcf8-b3c5-4b38-a2c7-1319b21e1481 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:13.457245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880fcf8-b3c5-4b38-a2c7-1319b21e1481 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:13.458086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880fcf8-b3c5-4b38-a2c7-1319b21e1481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:13.515123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3880fcf8-b3c5-4b38-a2c7-1319b21e1481 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:13.518318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3880fcf8-b3c5-4b38-a2c7-1319b21e1481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:13.686916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 14/25] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:14.158892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de39557-a15d-47ae-bc28-4cab19e235ba None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:14.175595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ea86a-c1e7-4aa0-aad7-e00619f53aa0 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:14.176032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ea86a-c1e7-4aa0-aad7-e00619f53aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:14.176274 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ea86a-c1e7-4aa0-aad7-e00619f53aa0 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:14.178475 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 15/26] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:07:14.208045 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c66aa6-bdc8-4984-8f86-9fdee1c78dc2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:14.208428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c66aa6-bdc8-4984-8f86-9fdee1c78dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:14.208575 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c66aa6-bdc8-4984-8f86-9fdee1c78dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:14.250391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-00c66aa6-bdc8-4984-8f86-9fdee1c78dc2 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:14.251705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-00c66aa6-bdc8-4984-8f86-9fdee1c78dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:14.283752 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:14.418120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de39557-a15d-47ae-bc28-4cab19e235ba None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:14.432509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 16/27] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:07:14.433981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f33ee2-ff56-409e-a92e-5117bd45516f None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:14.434308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f33ee2-ff56-409e-a92e-5117bd45516f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:14.434641 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f33ee2-ff56-409e-a92e-5117bd45516f None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:14.436573 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 17/28] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:14.437382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f33ee2-ff56-409e-a92e-5117bd45516f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:14.442439 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:14.518687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d20e2a-2cfa-4939-a410-1029a54edfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:14.519456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d20e2a-2cfa-4939-a410-1029a54edfac None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:14.519746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d20e2a-2cfa-4939-a410-1029a54edfac None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:14.520025 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d20e2a-2cfa-4939-a410-1029a54edfac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:14.551203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d20e2a-2cfa-4939-a410-1029a54edfac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:14.565769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f33ee2-ff56-409e-a92e-5117bd45516f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-89d20e2a-2cfa-4939-a410-1029a54edfac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:14.568834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:14.570315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:14.571691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 12/29] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:07:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:14.573701 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bdb88-8df5-4b3e-b12f-3cd10dbe4277 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:14.573701 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bdb88-8df5-4b3e-b12f-3cd10dbe4277 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:14.574352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bdb88-8df5-4b3e-b12f-3cd10dbe4277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:14.607567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253bb06-d54f-4040-b7cb-04fa6ab89644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:14.608238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253bb06-d54f-4040-b7cb-04fa6ab89644 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:14.608520 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253bb06-d54f-4040-b7cb-04fa6ab89644 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:14.608792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253bb06-d54f-4040-b7cb-04fa6ab89644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:14.621040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253bb06-d54f-4040-b7cb-04fa6ab89644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:14.622583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253bb06-d54f-4040-b7cb-04fa6ab89644 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6253bb06-d54f-4040-b7cb-04fa6ab89644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:07:14.642479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:14.644123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:07:14.645635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:07:14.645635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:07:14.645635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:07:14.645635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 18/30] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:14.645635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253bb06-d54f-4040-b7cb-04fa6ab89644 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:14.648837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fa4bdb88-8df5-4b3e-b12f-3cd10dbe4277 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:14.651404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fa4bdb88-8df5-4b3e-b12f-3cd10dbe4277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:14.747566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:14.809443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253bb06-d54f-4040-b7cb-04fa6ab89644 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:14.816268 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 13/31] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:14.817128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bdb88-8df5-4b3e-b12f-3cd10dbe4277 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:14.882090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:14.904462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726b39a-3449-492d-9842-0e625f279608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:14.905087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726b39a-3449-492d-9842-0e625f279608 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:14.905390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726b39a-3449-492d-9842-0e625f279608 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:14.905643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726b39a-3449-492d-9842-0e625f279608 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:14.906358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726b39a-3449-492d-9842-0e625f279608 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:14.906667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726b39a-3449-492d-9842-0e625f279608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:14.908642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726b39a-3449-492d-9842-0e625f279608 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:14.932099 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 19/32] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:14.932672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726b39a-3449-492d-9842-0e625f279608 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:14.997162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bdb88-8df5-4b3e-b12f-3cd10dbe4277 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:15.075692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e4ba4-c985-4b05-bf6e-490688007fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:15.075692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e4ba4-c985-4b05-bf6e-490688007fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:15.075692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e4ba4-c985-4b05-bf6e-490688007fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:15.075692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e4ba4-c985-4b05-bf6e-490688007fc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:15.076649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e4ba4-c985-4b05-bf6e-490688007fc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:15.077189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e4ba4-c985-4b05-bf6e-490688007fc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:15.080782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e4ba4-c985-4b05-bf6e-490688007fc8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:15.113272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 14/33] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:15.116478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e4ba4-c985-4b05-bf6e-490688007fc8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:15.193118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726b39a-3449-492d-9842-0e625f279608 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:15.231029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:15.272358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7953a70-fa66-4dfe-8b9e-d76f8840122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:15.273970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7953a70-fa66-4dfe-8b9e-d76f8840122c None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:15.274490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7953a70-fa66-4dfe-8b9e-d76f8840122c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:15.274998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7953a70-fa66-4dfe-8b9e-d76f8840122c None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:15.277362 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7953a70-fa66-4dfe-8b9e-d76f8840122c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:15.280166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7953a70-fa66-4dfe-8b9e-d76f8840122c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:15.330539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:15.345812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e4ba4-c985-4b05-bf6e-490688007fc8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:15.413816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76907-1855-4770-8ae9-28ab834c1fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:15.414853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76907-1855-4770-8ae9-28ab834c1fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:15.415317 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76907-1855-4770-8ae9-28ab834c1fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:15.415762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d76907-1855-4770-8ae9-28ab834c1fe6 None admin] PATH_INFO: `/v3/users/b9a31b5d481f471a90714fa648bfed35` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:15.418405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76907-1855-4770-8ae9-28ab834c1fe6 None admin] RBAC: Authorizing `identity:get_user(user_id=b9a31b5d481f471a90714fa648bfed35)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:15.421112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d76907-1855-4770-8ae9-28ab834c1fe6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:15.424381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 15/34] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:15 2022] GET /identity/v3/users/b9a31b5d481f471a90714fa648bfed35 => generated 247 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:15.426256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76907-1855-4770-8ae9-28ab834c1fe6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:15.438636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 20/35] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Sep 21 16:07:14 2022] POST /identity/v3/roles => generated 225 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:07:15.441907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045ff84-6580-4300-be28-db2e6b52c08b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:15.442934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045ff84-6580-4300-be28-db2e6b52c08b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:15.443573 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b045ff84-6580-4300-be28-db2e6b52c08b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:15.447518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 21/36] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:15.451705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045ff84-6580-4300-be28-db2e6b52c08b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:15.650712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d76907-1855-4770-8ae9-28ab834c1fe6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:15.691188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b045ff84-6580-4300-be28-db2e6b52c08b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:15.702674 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34d9bc-d23f-451f-9aec-06c7bda8a4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:15.703299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34d9bc-d23f-451f-9aec-06c7bda8a4e7 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:15.703580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34d9bc-d23f-451f-9aec-06c7bda8a4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:15.703855 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34d9bc-d23f-451f-9aec-06c7bda8a4e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:15.705336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34d9bc-d23f-451f-9aec-06c7bda8a4e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:15.706804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34d9bc-d23f-451f-9aec-06c7bda8a4e7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:15.763736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd66d89-3d15-4392-bc3b-ad330bd27122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:15.764804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd66d89-3d15-4392-bc3b-ad330bd27122 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:15.765305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd66d89-3d15-4392-bc3b-ad330bd27122 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:15.765805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd66d89-3d15-4392-bc3b-ad330bd27122 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:15.767716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd66d89-3d15-4392-bc3b-ad330bd27122 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:15.770740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd66d89-3d15-4392-bc3b-ad330bd27122 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:15.796503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 22/37] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Sep 21 16:07:15 2022] POST /identity/v3/roles => generated 219 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:07:15.799404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd66d89-3d15-4392-bc3b-ad330bd27122 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:15.854969 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 16/38] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:07:15 2022] POST /identity/v3/projects => generated 301 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:07:15.856574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7b033-31a6-4375-89db-766d29a74ef1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:15.856868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7b033-31a6-4375-89db-766d29a74ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:15.857279 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c7b033-31a6-4375-89db-766d29a74ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:15.879951 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b8c7b033-31a6-4375-89db-766d29a74ef1 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:15.881523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b8c7b033-31a6-4375-89db-766d29a74ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:15.897011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:15.900910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:16.023155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 17/39] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:16.026147 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eeef74-e03f-4665-806c-5817ec459a7a None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:16.026970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eeef74-e03f-4665-806c-5817ec459a7a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:16.027643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eeef74-e03f-4665-806c-5817ec459a7a None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:16.030096 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 18/40] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:16.040160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a2927-1a2e-4b2f-9ebe-bd11632d2ae0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:16.040846 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a2927-1a2e-4b2f-9ebe-bd11632d2ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:16.041429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4a2927-1a2e-4b2f-9ebe-bd11632d2ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:16.068729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd66d89-3d15-4392-bc3b-ad330bd27122 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:16.072183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9e4a2927-1a2e-4b2f-9ebe-bd11632d2ae0 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:16.074540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9e4a2927-1a2e-4b2f-9ebe-bd11632d2ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:16.136512 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b25e5c-ed86-4306-9539-63da3b63dce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:16.137934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b25e5c-ed86-4306-9539-63da3b63dce7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:16.138451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b25e5c-ed86-4306-9539-63da3b63dce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:16.139087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b25e5c-ed86-4306-9539-63da3b63dce7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:16.141577 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b25e5c-ed86-4306-9539-63da3b63dce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:16.144553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b25e5c-ed86-4306-9539-63da3b63dce7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:16.147385 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 23/41] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:16.157051 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b25e5c-ed86-4306-9539-63da3b63dce7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:16.209117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 19/42] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:07:16.371360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b25e5c-ed86-4306-9539-63da3b63dce7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:16.488766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a2927-1a2e-4b2f-9ebe-bd11632d2ae0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:16.536415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:16.580693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57a5e5-3083-47a3-aeea-a5e01fae239f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:16.581776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57a5e5-3083-47a3-aeea-a5e01fae239f None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:16.582288 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57a5e5-3083-47a3-aeea-a5e01fae239f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:16.582769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57a5e5-3083-47a3-aeea-a5e01fae239f None admin] PATH_INFO: `/v3/domains/default/users/b9a31b5d481f471a90714fa648bfed35/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:16.585627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57a5e5-3083-47a3-aeea-a5e01fae239f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=b9a31b5d481f471a90714fa648bfed35, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:16.588192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57a5e5-3083-47a3-aeea-a5e01fae239f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:16.594472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:16.609538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 24/43] 10.222.0.64 () {58 vars in 1361 bytes} [Wed Sep 21 16:07:16 2022] PUT /identity/v3/domains/default/users/b9a31b5d481f471a90714fa648bfed35/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:07:16.615329 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57a5e5-3083-47a3-aeea-a5e01fae239f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:16.636168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4a2927-1a2e-4b2f-9ebe-bd11632d2ae0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:16.699387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ef523-a743-4be3-874d-ca645daf1683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:16.700223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ef523-a743-4be3-874d-ca645daf1683 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:16.700682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ef523-a743-4be3-874d-ca645daf1683 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:16.701093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ef523-a743-4be3-874d-ca645daf1683 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:16.702684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ef523-a743-4be3-874d-ca645daf1683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:16.705323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ef523-a743-4be3-874d-ca645daf1683 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:16.707635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 20/44] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:16.714028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ef523-a743-4be3-874d-ca645daf1683 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:16.795089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57a5e5-3083-47a3-aeea-a5e01fae239f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:16.852701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9af732-d6cf-4897-871a-ffe0e8de7674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:16.853553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9af732-d6cf-4897-871a-ffe0e8de7674 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:16.853954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9af732-d6cf-4897-871a-ffe0e8de7674 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:16.854373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9af732-d6cf-4897-871a-ffe0e8de7674 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:16.855546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9af732-d6cf-4897-871a-ffe0e8de7674 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:16.857984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9af732-d6cf-4897-871a-ffe0e8de7674 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:16.882510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 25/45] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Sep 21 16:07:16 2022] POST /identity/v3/roles => generated 223 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:07:16.898535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ef523-a743-4be3-874d-ca645daf1683 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:16.943895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67d972-e7aa-4b34-b41c-ee3f84c85aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:16.944649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67d972-e7aa-4b34-b41c-ee3f84c85aed None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:16.945014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67d972-e7aa-4b34-b41c-ee3f84c85aed None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:16.945380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67d972-e7aa-4b34-b41c-ee3f84c85aed None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:16.946241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67d972-e7aa-4b34-b41c-ee3f84c85aed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:16.947949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67d972-e7aa-4b34-b41c-ee3f84c85aed None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:16.968111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:16.985727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 21/46] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:07:16 2022] POST /identity/v3/projects => generated 312 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:07:20.188328 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478ade5-2d6d-4c1c-942a-ccb1f8fc133d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:20.189797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478ade5-2d6d-4c1c-942a-ccb1f8fc133d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:20.189937 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478ade5-2d6d-4c1c-942a-ccb1f8fc133d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:20.191610 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 26/47] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:20.197530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383b0c3-a95c-4419-8d39-0d02ff0abf66 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:20.198186 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383b0c3-a95c-4419-8d39-0d02ff0abf66 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:20.198331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383b0c3-a95c-4419-8d39-0d02ff0abf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:20.218650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5383b0c3-a95c-4419-8d39-0d02ff0abf66 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:20.220462 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5383b0c3-a95c-4419-8d39-0d02ff0abf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:20.332709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 22/48] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:20.434575 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79fb44-f92e-496e-8c15-11c8202513ac None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:20.434867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79fb44-f92e-496e-8c15-11c8202513ac None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:20.435165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79fb44-f92e-496e-8c15-11c8202513ac None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:20.436521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 27/49] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:20.442735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8613a3f-e043-472d-8f1d-40bff7824c50 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:20.443002 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8613a3f-e043-472d-8f1d-40bff7824c50 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:20.443292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8613a3f-e043-472d-8f1d-40bff7824c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:20.464152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d8613a3f-e043-472d-8f1d-40bff7824c50 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:20.465387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d8613a3f-e043-472d-8f1d-40bff7824c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:20.540150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 23/50] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:20.608425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79fb44-f92e-496e-8c15-11c8202513ac None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:20.638559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:20.724286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79fb44-f92e-496e-8c15-11c8202513ac None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:20.758611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fb46f-efd1-418c-b08d-a12695ffc59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:20.759253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fb46f-efd1-418c-b08d-a12695ffc59a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:20.759538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fb46f-efd1-418c-b08d-a12695ffc59a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:20.759817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fb46f-efd1-418c-b08d-a12695ffc59a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:20.770338 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fb46f-efd1-418c-b08d-a12695ffc59a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-8a2fb46f-efd1-418c-b08d-a12695ffc59a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:20.775237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:20.776074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:07:20.776916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 28/51] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:07:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:20.782227 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8613a3f-e043-472d-8f1d-40bff7824c50 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:20.808018 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fb46f-efd1-418c-b08d-a12695ffc59a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:20.839493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:20.885525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8613a3f-e043-472d-8f1d-40bff7824c50 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:20.896244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:20.912896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fb46f-efd1-418c-b08d-a12695ffc59a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:20.922122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded9fe8-2572-409b-9c73-790f835b896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:20.923292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded9fe8-2572-409b-9c73-790f835b896e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:20.923577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded9fe8-2572-409b-9c73-790f835b896e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:20.923856 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded9fe8-2572-409b-9c73-790f835b896e None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:20.924538 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded9fe8-2572-409b-9c73-790f835b896e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:20.924843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded9fe8-2572-409b-9c73-790f835b896e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:20.926265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded9fe8-2572-409b-9c73-790f835b896e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:20.939272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 24/52] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:20.945045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded9fe8-2572-409b-9c73-790f835b896e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:20.952334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a1e31-d4a9-42ba-9565-5e3eef9ab28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:20.953080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a1e31-d4a9-42ba-9565-5e3eef9ab28d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:20.953280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a1e31-d4a9-42ba-9565-5e3eef9ab28d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:20.953557 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5a1e31-d4a9-42ba-9565-5e3eef9ab28d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:20.955783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a1e31-d4a9-42ba-9565-5e3eef9ab28d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:20.957261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5a1e31-d4a9-42ba-9565-5e3eef9ab28d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:20.958895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 29/53] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:20.964483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a1e31-d4a9-42ba-9565-5e3eef9ab28d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:21.055436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded9fe8-2572-409b-9c73-790f835b896e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:21.074466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:21.083818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5a1e31-d4a9-42ba-9565-5e3eef9ab28d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:21.096763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cf4da-3a30-43d5-a4bc-3971041c6113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:21.097492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cf4da-3a30-43d5-a4bc-3971041c6113 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:21.098010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cf4da-3a30-43d5-a4bc-3971041c6113 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:21.098010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cf4da-3a30-43d5-a4bc-3971041c6113 None admin] PATH_INFO: `/v3/users/b9a31b5d481f471a90714fa648bfed35` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:21.099223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cf4da-3a30-43d5-a4bc-3971041c6113 None admin] RBAC: Authorizing `identity:get_user(user_id=b9a31b5d481f471a90714fa648bfed35)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:21.100681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cf4da-3a30-43d5-a4bc-3971041c6113 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:21.102486 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 25/54] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:20 2022] GET /identity/v3/users/b9a31b5d481f471a90714fa648bfed35 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:21.108028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cf4da-3a30-43d5-a4bc-3971041c6113 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:21.126456 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b95914-ac4f-4699-a923-87602436a318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:21.126906 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b95914-ac4f-4699-a923-87602436a318 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:21.127191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b95914-ac4f-4699-a923-87602436a318 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:21.127463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b95914-ac4f-4699-a923-87602436a318 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:21.128141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b95914-ac4f-4699-a923-87602436a318 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:21.129596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b95914-ac4f-4699-a923-87602436a318 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:21.161561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 30/55] 10.222.0.64 () {60 vars in 1140 bytes} [Wed Sep 21 16:07:20 2022] POST /identity/v3/projects => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:07:21.333444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cf4da-3a30-43d5-a4bc-3971041c6113 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:21.372018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7856d52-cad5-4188-a4e8-e592286f718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:21.372633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7856d52-cad5-4188-a4e8-e592286f718f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:21.372911 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7856d52-cad5-4188-a4e8-e592286f718f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:21.373222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7856d52-cad5-4188-a4e8-e592286f718f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:21.374519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7856d52-cad5-4188-a4e8-e592286f718f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:21.375969 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7856d52-cad5-4188-a4e8-e592286f718f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:21.382122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 26/56] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:21.383700 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b95914-ac4f-4699-a923-87602436a318 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:21.393681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:21.505455 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b95914-ac4f-4699-a923-87602436a318 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:21.545977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abdbdf-cc45-4811-8a96-d63ce4ced4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:21.546984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abdbdf-cc45-4811-8a96-d63ce4ced4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:21.547459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abdbdf-cc45-4811-8a96-d63ce4ced4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:21.548031 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abdbdf-cc45-4811-8a96-d63ce4ced4c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:21.549476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abdbdf-cc45-4811-8a96-d63ce4ced4c0 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.domain.id=default, user.id=b9a31b5d481f471a90714fa648bfed35) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:21.549912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abdbdf-cc45-4811-8a96-d63ce4ced4c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:21.551563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abdbdf-cc45-4811-8a96-d63ce4ced4c0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:21.558186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 31/57] 10.222.0.64 () {56 vars in 1322 bytes} [Wed Sep 21 16:07:21 2022] GET /identity/v3/role_assignments?user.id=b9a31b5d481f471a90714fa648bfed35&scope.domain.id=default&role.id=459d01a218314543a7a8329e88b78331 => generated 530 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:23.386027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740608b-dead-405c-80f3-35945ead74cc None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:23.386568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740608b-dead-405c-80f3-35945ead74cc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:23.386673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740608b-dead-405c-80f3-35945ead74cc None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:23.387926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 27/58] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:23.393664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf9320-897f-4088-a90f-0a59d3f557d5 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:23.394029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf9320-897f-4088-a90f-0a59d3f557d5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:23.394293 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf9320-897f-4088-a90f-0a59d3f557d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:23.412486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6bbf9320-897f-4088-a90f-0a59d3f557d5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:23.414018 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6bbf9320-897f-4088-a90f-0a59d3f557d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:23.475579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:23.480559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 32/59] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:23.749064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740608b-dead-405c-80f3-35945ead74cc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:23.805490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:23.851468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740608b-dead-405c-80f3-35945ead74cc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:23.883490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407fce75-ce11-4c36-8ac6-23275b05fadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:23.884117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407fce75-ce11-4c36-8ac6-23275b05fadb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:23.884400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407fce75-ce11-4c36-8ac6-23275b05fadb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:23.884676 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407fce75-ce11-4c36-8ac6-23275b05fadb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:23.885825 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407fce75-ce11-4c36-8ac6-23275b05fadb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:23.887329 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407fce75-ce11-4c36-8ac6-23275b05fadb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:23.888974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 28/60] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:23.895503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf9320-897f-4088-a90f-0a59d3f557d5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:23.996204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf9320-897f-4088-a90f-0a59d3f557d5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:24.030662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a760b-c498-4338-acf8-9ac05f0c462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:24.031199 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a760b-c498-4338-acf8-9ac05f0c462f None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:24.031485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a760b-c498-4338-acf8-9ac05f0c462f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:24.031765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a760b-c498-4338-acf8-9ac05f0c462f None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:24.032455 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a760b-c498-4338-acf8-9ac05f0c462f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:24.033896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a760b-c498-4338-acf8-9ac05f0c462f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:24.060247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 33/61] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:07:23 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:07:27.713051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bfed8-ad63-4826-9069-44d2c851458d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:27.714581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bfed8-ad63-4826-9069-44d2c851458d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:27.715316 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bfed8-ad63-4826-9069-44d2c851458d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:27.718177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 29/62] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:27.730328 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fcdab-39f4-4f7f-b9be-ab8d3c639490 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:27.731284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fcdab-39f4-4f7f-b9be-ab8d3c639490 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:27.731767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fcdab-39f4-4f7f-b9be-ab8d3c639490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:27.781784 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-419fcdab-39f4-4f7f-b9be-ab8d3c639490 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:27.783848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-419fcdab-39f4-4f7f-b9be-ab8d3c639490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:27.804950 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861a55a-5328-4157-b252-168fae2794af None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:27.805601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861a55a-5328-4157-b252-168fae2794af None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:27.806101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861a55a-5328-4157-b252-168fae2794af None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:27.808031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 30/63] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:27.814770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121db91-3c42-491f-9312-a649da9e1610 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:27.815396 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121db91-3c42-491f-9312-a649da9e1610 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:27.815890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121db91-3c42-491f-9312-a649da9e1610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:27.839407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2121db91-3c42-491f-9312-a649da9e1610 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:27.840936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2121db91-3c42-491f-9312-a649da9e1610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:27.876025 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:27.906443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 34/64] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:07:27.908733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b807f-2cad-479d-ac8a-70c3daae9c65 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:27.909246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b807f-2cad-479d-ac8a-70c3daae9c65 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:27.909746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b807f-2cad-479d-ac8a-70c3daae9c65 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:27.912446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 35/65] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:27.920056 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a35d4-2564-46c8-a776-874e37503a6b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:27.920560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a35d4-2564-46c8-a776-874e37503a6b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:27.921058 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a35d4-2564-46c8-a776-874e37503a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:27.967084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-da5a35d4-2564-46c8-a776-874e37503a6b None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:27.969377 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-da5a35d4-2564-46c8-a776-874e37503a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:28.007935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 31/66] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:28.108409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 36/67] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:07:28.248670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60edd3c-cd0d-4283-b561-2131b73072de None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:28.248953 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60edd3c-cd0d-4283-b561-2131b73072de None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:28.249487 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60edd3c-cd0d-4283-b561-2131b73072de None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:28.251839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 32/68] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:28.257758 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b22df-4f29-4817-9f07-591fa948fa0f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:28.258076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b22df-4f29-4817-9f07-591fa948fa0f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:28.258359 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b22df-4f29-4817-9f07-591fa948fa0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:28.287392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-116b22df-4f29-4817-9f07-591fa948fa0f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:28.288649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-116b22df-4f29-4817-9f07-591fa948fa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:28.330947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60edd3c-cd0d-4283-b561-2131b73072de None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:28.412064 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 37/69] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:28.415045 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b22df-4f29-4817-9f07-591fa948fa0f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:28.451521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:28.474200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:28.539702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60edd3c-cd0d-4283-b561-2131b73072de None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:28.542305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b22df-4f29-4817-9f07-591fa948fa0f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:28.582029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f78094-99c5-4c16-8030-d99699c24bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:28.582591 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01943d80-e901-45ae-aa93-c8f40f6c9519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:28.583233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f78094-99c5-4c16-8030-d99699c24bb2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:28.583673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01943d80-e901-45ae-aa93-c8f40f6c9519 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:28.583888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f78094-99c5-4c16-8030-d99699c24bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:28.584184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01943d80-e901-45ae-aa93-c8f40f6c9519 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:28.584512 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f78094-99c5-4c16-8030-d99699c24bb2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:28.584701 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01943d80-e901-45ae-aa93-c8f40f6c9519 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:28.596879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f78094-99c5-4c16-8030-d99699c24bb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-65f78094-99c5-4c16-8030-d99699c24bb2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:28.602226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:28.603070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:28.603810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:28.603810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:28.603810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:28.603810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:28.603810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:28.603810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:28.603810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:28.603810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:28.603810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:28.603810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:28.603810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:07:28.603810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:07:28.604976 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01943d80-e901-45ae-aa93-c8f40f6c9519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:28.605847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 38/70] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:07:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:28.607733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab87f9-6082-4cbc-ab6f-dd2c1c76c7d5 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:28.608155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab87f9-6082-4cbc-ab6f-dd2c1c76c7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:28.608558 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab87f9-6082-4cbc-ab6f-dd2c1c76c7d5 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:28.610562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 39/71] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:28.614956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab87f9-6082-4cbc-ab6f-dd2c1c76c7d5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-01943d80-e901-45ae-aa93-c8f40f6c9519 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:28.616382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:28.617288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:28.618079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:28.618079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:28.618079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:28.618079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:28.618079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:28.618079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:28.618079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:28.618079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:28.618079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:28.618079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:28.618079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:28.618079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:28.618669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 33/72] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:07:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:28.623792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01943d80-e901-45ae-aa93-c8f40f6c9519 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:28.735667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab87f9-6082-4cbc-ab6f-dd2c1c76c7d5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:28.748253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:28.779356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01943d80-e901-45ae-aa93-c8f40f6c9519 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:28.788696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d601c5-17e6-4ca8-99ea-6f574730442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:28.789399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d601c5-17e6-4ca8-99ea-6f574730442e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:28.789733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d601c5-17e6-4ca8-99ea-6f574730442e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:28.790222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d601c5-17e6-4ca8-99ea-6f574730442e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:28.796435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:28.802788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d601c5-17e6-4ca8-99ea-6f574730442e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-02d601c5-17e6-4ca8-99ea-6f574730442e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:28.807776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:28.809010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 40/73] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:28.810466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74a8c0-d581-4412-9db9-4fde19b09037 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:28.811749 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74a8c0-d581-4412-9db9-4fde19b09037 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:28.811749 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74a8c0-d581-4412-9db9-4fde19b09037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:28.827796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45592988-f080-4797-96e7-fe3b8fc8eb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:28.829170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45592988-f080-4797-96e7-fe3b8fc8eb0d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:28.829456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45592988-f080-4797-96e7-fe3b8fc8eb0d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:28.829731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45592988-f080-4797-96e7-fe3b8fc8eb0d None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:28.830535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45592988-f080-4797-96e7-fe3b8fc8eb0d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:28.830942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45592988-f080-4797-96e7-fe3b8fc8eb0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:28.831061 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-fd74a8c0-d581-4412-9db9-4fde19b09037 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:28.832299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45592988-f080-4797-96e7-fe3b8fc8eb0d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:28.833096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fd74a8c0-d581-4412-9db9-4fde19b09037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:28.841420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 34/74] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:07:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:28.842037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45592988-f080-4797-96e7-fe3b8fc8eb0d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:28.911505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 41/75] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:28.912392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74a8c0-d581-4412-9db9-4fde19b09037 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:28.975759 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45592988-f080-4797-96e7-fe3b8fc8eb0d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:29.021372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47ce23-b09b-49ad-9f79-22c09af93ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:29.022226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47ce23-b09b-49ad-9f79-22c09af93ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:29.022641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47ce23-b09b-49ad-9f79-22c09af93ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:29.023042 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47ce23-b09b-49ad-9f79-22c09af93ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:29.023973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47ce23-b09b-49ad-9f79-22c09af93ba5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:29.024534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47ce23-b09b-49ad-9f79-22c09af93ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:29.026437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47ce23-b09b-49ad-9f79-22c09af93ba5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:29.028219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:29.036897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74a8c0-d581-4412-9db9-4fde19b09037 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:29.040128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 35/76] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:29.041335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47ce23-b09b-49ad-9f79-22c09af93ba5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:29.080561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde76bba-5d9d-4058-8723-6a0fbaa5b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:29.081215 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde76bba-5d9d-4058-8723-6a0fbaa5b097 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:29.081500 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde76bba-5d9d-4058-8723-6a0fbaa5b097 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:29.081782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde76bba-5d9d-4058-8723-6a0fbaa5b097 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:29.086981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:29.094612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde76bba-5d9d-4058-8723-6a0fbaa5b097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-dde76bba-5d9d-4058-8723-6a0fbaa5b097 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:29.099531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:29.100427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:29.101185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:29.101185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:29.101185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:29.101185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:29.101185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:29.101185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:29.101185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:29.101185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:29.101185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:29.101185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:29.101185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:29.101185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:07:29.103099 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 42/77] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:29.104028 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde76bba-5d9d-4058-8723-6a0fbaa5b097 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:29.180096 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47ce23-b09b-49ad-9f79-22c09af93ba5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:29.225785 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e075017-0b71-4030-bdb0-d8572aa8b7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:29.226823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e075017-0b71-4030-bdb0-d8572aa8b7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:29.227301 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e075017-0b71-4030-bdb0-d8572aa8b7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:29.227747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e075017-0b71-4030-bdb0-d8572aa8b7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:29.228003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde76bba-5d9d-4058-8723-6a0fbaa5b097 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:29.228879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e075017-0b71-4030-bdb0-d8572aa8b7b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:29.229376 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e075017-0b71-4030-bdb0-d8572aa8b7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:29.231819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e075017-0b71-4030-bdb0-d8572aa8b7b8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:29.244367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 36/78] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:07:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:29.245246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e075017-0b71-4030-bdb0-d8572aa8b7b8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:29.275277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095deec2-bda7-4e9c-9543-976619f85d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:29.275831 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095deec2-bda7-4e9c-9543-976619f85d54 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:29.276114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095deec2-bda7-4e9c-9543-976619f85d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:29.276384 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095deec2-bda7-4e9c-9543-976619f85d54 None admin] PATH_INFO: `/v3/users/1888f4b82c7940fdb9abfe0a57b404c5` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:29.291028 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095deec2-bda7-4e9c-9543-976619f85d54 None admin] RBAC: Authorizing `identity:get_user(user_id=1888f4b82c7940fdb9abfe0a57b404c5)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:29.292493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095deec2-bda7-4e9c-9543-976619f85d54 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:29.294279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 43/79] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:29 2022] GET /identity/v3/users/1888f4b82c7940fdb9abfe0a57b404c5 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:29.294958 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095deec2-bda7-4e9c-9543-976619f85d54 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:29.313810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:29.352102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:29.356348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e075017-0b71-4030-bdb0-d8572aa8b7b8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:29.395595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60525a7d-966a-4556-be92-bbee6dbd9505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:29.396237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60525a7d-966a-4556-be92-bbee6dbd9505 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:29.396522 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60525a7d-966a-4556-be92-bbee6dbd9505 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:29.396799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60525a7d-966a-4556-be92-bbee6dbd9505 None admin] PATH_INFO: `/v3/users/b9a31b5d481f471a90714fa648bfed35` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:29.397985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60525a7d-966a-4556-be92-bbee6dbd9505 None admin] RBAC: Authorizing `identity:get_user(user_id=b9a31b5d481f471a90714fa648bfed35)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:29.399443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60525a7d-966a-4556-be92-bbee6dbd9505 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:29.401069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 37/80] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:29 2022] GET /identity/v3/users/b9a31b5d481f471a90714fa648bfed35 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:29.401872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60525a7d-966a-4556-be92-bbee6dbd9505 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:29.415784 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095deec2-bda7-4e9c-9543-976619f85d54 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:29.463244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2b1c9-8b76-49a0-8a42-003327faa790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:29.463879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2b1c9-8b76-49a0-8a42-003327faa790 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:29.464162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2b1c9-8b76-49a0-8a42-003327faa790 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:29.464440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb2b1c9-8b76-49a0-8a42-003327faa790 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:29.465144 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2b1c9-8b76-49a0-8a42-003327faa790 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:29.465457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2b1c9-8b76-49a0-8a42-003327faa790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:29.466906 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb2b1c9-8b76-49a0-8a42-003327faa790 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:29.476421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 44/81] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:07:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:29.477376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2b1c9-8b76-49a0-8a42-003327faa790 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:29.528632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60525a7d-966a-4556-be92-bbee6dbd9505 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:29.568463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:29.573598 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0fda4-9a7b-4ee6-b7af-bc97b6587a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:29.574431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0fda4-9a7b-4ee6-b7af-bc97b6587a4c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:29.574803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0fda4-9a7b-4ee6-b7af-bc97b6587a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:29.575212 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0fda4-9a7b-4ee6-b7af-bc97b6587a4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:29.576757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0fda4-9a7b-4ee6-b7af-bc97b6587a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:29.579313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0fda4-9a7b-4ee6-b7af-bc97b6587a4c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:29.581485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 38/82] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:29.583023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0fda4-9a7b-4ee6-b7af-bc97b6587a4c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:29.601080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb2b1c9-8b76-49a0-8a42-003327faa790 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:29.624407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:29.636988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b99401-e70a-42c3-9113-cb4a13fc7831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:29.637642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b99401-e70a-42c3-9113-cb4a13fc7831 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:29.637933 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b99401-e70a-42c3-9113-cb4a13fc7831 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:29.638245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b99401-e70a-42c3-9113-cb4a13fc7831 None admin] PATH_INFO: `/v3/users/1888f4b82c7940fdb9abfe0a57b404c5` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:29.639434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b99401-e70a-42c3-9113-cb4a13fc7831 None admin] RBAC: Authorizing `identity:get_user(user_id=1888f4b82c7940fdb9abfe0a57b404c5)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:29.641494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b99401-e70a-42c3-9113-cb4a13fc7831 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:29.643175 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 45/83] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:29 2022] GET /identity/v3/users/1888f4b82c7940fdb9abfe0a57b404c5 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:29.643984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b99401-e70a-42c3-9113-cb4a13fc7831 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:29.701034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0fda4-9a7b-4ee6-b7af-bc97b6587a4c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:29.737855 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb7115-5f85-4034-98ef-9c350013861c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:29.738824 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb7115-5f85-4034-98ef-9c350013861c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:29.739256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb7115-5f85-4034-98ef-9c350013861c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:29.739668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb7115-5f85-4034-98ef-9c350013861c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:29.758010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b99401-e70a-42c3-9113-cb4a13fc7831 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:29.795640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374fa19-50ee-4335-8598-ec14972411fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:29.796189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374fa19-50ee-4335-8598-ec14972411fd None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:29.796475 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374fa19-50ee-4335-8598-ec14972411fd None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:29.796756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374fa19-50ee-4335-8598-ec14972411fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:29.813605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:29.820303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb7115-5f85-4034-98ef-9c350013861c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:29.822605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb7115-5f85-4034-98ef-9c350013861c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:29.860152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:29.876881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374fa19-50ee-4335-8598-ec14972411fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:29.878415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374fa19-50ee-4335-8598-ec14972411fd None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:29.947760 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 39/84] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:07:29.948935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb7115-5f85-4034-98ef-9c350013861c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:29.998243 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 46/85] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:07:29.999358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374fa19-50ee-4335-8598-ec14972411fd None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:30.049020 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:30.065811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb7115-5f85-4034-98ef-9c350013861c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:30.103236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d6778-7449-479a-b6c1-8321e6b06565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:30.103875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d6778-7449-479a-b6c1-8321e6b06565 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:30.104151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d6778-7449-479a-b6c1-8321e6b06565 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:30.104422 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d6778-7449-479a-b6c1-8321e6b06565 None admin] PATH_INFO: `/v3/users/1888f4b82c7940fdb9abfe0a57b404c5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:30.105620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d6778-7449-479a-b6c1-8321e6b06565 None admin] RBAC: Authorizing `identity:get_user(user_id=1888f4b82c7940fdb9abfe0a57b404c5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:30.107085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d6778-7449-479a-b6c1-8321e6b06565 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:30.109170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 40/86] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:29 2022] GET /identity/v3/users/1888f4b82c7940fdb9abfe0a57b404c5 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:30.110494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d6778-7449-479a-b6c1-8321e6b06565 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:30.113314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:30.118980 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374fa19-50ee-4335-8598-ec14972411fd None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:30.162536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09652c3b-873c-4b29-bf04-154710d1ecd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:30.163156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09652c3b-873c-4b29-bf04-154710d1ecd9 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:30.163446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09652c3b-873c-4b29-bf04-154710d1ecd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:30.163721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09652c3b-873c-4b29-bf04-154710d1ecd9 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:30.164390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09652c3b-873c-4b29-bf04-154710d1ecd9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:30.165867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09652c3b-873c-4b29-bf04-154710d1ecd9 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:30.193824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 47/87] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:07:29 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:07:30.195212 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09652c3b-873c-4b29-bf04-154710d1ecd9 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:30.246019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d6778-7449-479a-b6c1-8321e6b06565 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:30.291301 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592ec68-4ee2-4e3a-a248-88543ed1735c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:30.291970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592ec68-4ee2-4e3a-a248-88543ed1735c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:30.292225 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592ec68-4ee2-4e3a-a248-88543ed1735c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:30.292503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592ec68-4ee2-4e3a-a248-88543ed1735c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:30.367979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:30.373354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09652c3b-873c-4b29-bf04-154710d1ecd9 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:30.403467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592ec68-4ee2-4e3a-a248-88543ed1735c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:30.405096 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592ec68-4ee2-4e3a-a248-88543ed1735c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:30.407835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:30.427118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0c8f6-ac25-4e5c-8361-7db9df8ee510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:30.427774 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0c8f6-ac25-4e5c-8361-7db9df8ee510 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:30.428055 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0c8f6-ac25-4e5c-8361-7db9df8ee510 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:30.428329 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0c8f6-ac25-4e5c-8361-7db9df8ee510 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:30.429578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0c8f6-ac25-4e5c-8361-7db9df8ee510 None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:30.431634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0c8f6-ac25-4e5c-8361-7db9df8ee510 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:30.433540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 48/88] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:30 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:30.435736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0c8f6-ac25-4e5c-8361-7db9df8ee510 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:30.563887 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 41/89] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:07:30.564607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592ec68-4ee2-4e3a-a248-88543ed1735c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:30.597986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0c8f6-ac25-4e5c-8361-7db9df8ee510 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:30.657925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747400f-bb97-475a-aa2a-07055f804629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:30.659338 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747400f-bb97-475a-aa2a-07055f804629 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:30.659695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747400f-bb97-475a-aa2a-07055f804629 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:30.660246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a747400f-bb97-475a-aa2a-07055f804629 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:30.661909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747400f-bb97-475a-aa2a-07055f804629 None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:30.665515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a747400f-bb97-475a-aa2a-07055f804629 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:30.667624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 49/90] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:30 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:30.668450 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747400f-bb97-475a-aa2a-07055f804629 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:30.725765 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592ec68-4ee2-4e3a-a248-88543ed1735c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:30.728909 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:30.745899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:30.784687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525146b-4d12-44a9-a763-bd607767425d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:30.785633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525146b-4d12-44a9-a763-bd607767425d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:30.785934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525146b-4d12-44a9-a763-bd607767425d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:30.786264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525146b-4d12-44a9-a763-bd607767425d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:30.837926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a747400f-bb97-475a-aa2a-07055f804629 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:30.886622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525146b-4d12-44a9-a763-bd607767425d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:30.888169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525146b-4d12-44a9-a763-bd607767425d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:30.888449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23470bd-66f6-42e2-8a01-7536f6db1c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:30.889188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23470bd-66f6-42e2-8a01-7536f6db1c28 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:30.889579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23470bd-66f6-42e2-8a01-7536f6db1c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:30.890082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23470bd-66f6-42e2-8a01-7536f6db1c28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:30.891073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23470bd-66f6-42e2-8a01-7536f6db1c28 None admin] RBAC: Adding query filter params (role.id=aa320f864d914f65bebe5463994ff536, scope.project.id=832b79a184af40d7ad323b5e97816800, user.id=1888f4b82c7940fdb9abfe0a57b404c5) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:30.891534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23470bd-66f6-42e2-8a01-7536f6db1c28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:30.893498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23470bd-66f6-42e2-8a01-7536f6db1c28 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:30.906185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 50/91] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:30 2022] GET /identity/v3/role_assignments?user.id=1888f4b82c7940fdb9abfe0a57b404c5&scope.project.id=832b79a184af40d7ad323b5e97816800&role.id=aa320f864d914f65bebe5463994ff536 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:30.907893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23470bd-66f6-42e2-8a01-7536f6db1c28 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:31.135198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 42/92] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:07:31.136173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525146b-4d12-44a9-a763-bd607767425d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:31.139307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:31.158334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:31.162525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23470bd-66f6-42e2-8a01-7536f6db1c28 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:31.240968 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69632883-6259-40be-854f-59b8c555316e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:31.241646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69632883-6259-40be-854f-59b8c555316e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:31.241934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69632883-6259-40be-854f-59b8c555316e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:31.242245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69632883-6259-40be-854f-59b8c555316e None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:31.247909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69632883-6259-40be-854f-59b8c555316e None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:31.249446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69632883-6259-40be-854f-59b8c555316e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:31.257701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 51/93] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:30 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:31.258683 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69632883-6259-40be-854f-59b8c555316e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:31.345357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525146b-4d12-44a9-a763-bd607767425d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:31.396375 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddb9af-64db-48f3-972f-16e6da00e378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:31.396906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddb9af-64db-48f3-972f-16e6da00e378 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:31.397197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddb9af-64db-48f3-972f-16e6da00e378 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:31.397471 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddb9af-64db-48f3-972f-16e6da00e378 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:31.398261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddb9af-64db-48f3-972f-16e6da00e378 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=832b79a184af40d7ad323b5e97816800, user.id=b9a31b5d481f471a90714fa648bfed35) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:31.398648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddb9af-64db-48f3-972f-16e6da00e378 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:31.400137 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddb9af-64db-48f3-972f-16e6da00e378 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:31.418014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 43/94] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:31 2022] GET /identity/v3/role_assignments?user.id=b9a31b5d481f471a90714fa648bfed35&scope.project.id=832b79a184af40d7ad323b5e97816800&role.id=459d01a218314543a7a8329e88b78331 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:31.425412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddb9af-64db-48f3-972f-16e6da00e378 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:31.519465 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69632883-6259-40be-854f-59b8c555316e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:31.598380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:31.623038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc10389-4982-43ae-b0a8-a8a412b0a3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:31.623719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc10389-4982-43ae-b0a8-a8a412b0a3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:31.624008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc10389-4982-43ae-b0a8-a8a412b0a3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:31.624292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc10389-4982-43ae-b0a8-a8a412b0a3c8 None admin] PATH_INFO: `/v3/projects/de138855cc8744659425af5f269b8fbc` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:31.626268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc10389-4982-43ae-b0a8-a8a412b0a3c8 None admin] RBAC: Authorizing `identity:get_project(project_id=de138855cc8744659425af5f269b8fbc)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:31.627863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc10389-4982-43ae-b0a8-a8a412b0a3c8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:31.629991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 52/95] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:31 2022] GET /identity/v3/projects/de138855cc8744659425af5f269b8fbc => generated 312 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:31.636017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc10389-4982-43ae-b0a8-a8a412b0a3c8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:31.673563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddb9af-64db-48f3-972f-16e6da00e378 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:31.739698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c35904b-83f3-43a1-922b-9334a75cc7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:31.740353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35904b-83f3-43a1-922b-9334a75cc7fc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:31.740639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35904b-83f3-43a1-922b-9334a75cc7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:31.740919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35904b-83f3-43a1-922b-9334a75cc7fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:31.741682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35904b-83f3-43a1-922b-9334a75cc7fc None admin] RBAC: Adding query filter params (role.id=20378c15bf524a4c960b49d2c53cf6b2, scope.project.id=832b79a184af40d7ad323b5e97816800, user.id=1888f4b82c7940fdb9abfe0a57b404c5) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:31.742002 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35904b-83f3-43a1-922b-9334a75cc7fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:31.743449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:31.743624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c35904b-83f3-43a1-922b-9334a75cc7fc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:31.762949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 44/96] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:31 2022] GET /identity/v3/role_assignments?user.id=1888f4b82c7940fdb9abfe0a57b404c5&scope.project.id=832b79a184af40d7ad323b5e97816800&role.id=20378c15bf524a4c960b49d2c53cf6b2 => generated 259 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:31.876617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc10389-4982-43ae-b0a8-a8a412b0a3c8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:31.924718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4bb69-4298-4ba0-ac5a-ea3af6293936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:31.925252 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4bb69-4298-4ba0-ac5a-ea3af6293936 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:31.925535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4bb69-4298-4ba0-ac5a-ea3af6293936 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:31.925808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4bb69-4298-4ba0-ac5a-ea3af6293936 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:31.926684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4bb69-4298-4ba0-ac5a-ea3af6293936 None admin] RBAC: Adding query filter params (role.id=20378c15bf524a4c960b49d2c53cf6b2, scope.project.id=de138855cc8744659425af5f269b8fbc, user.id=1888f4b82c7940fdb9abfe0a57b404c5) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:31.926999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4bb69-4298-4ba0-ac5a-ea3af6293936 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:31.928469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4bb69-4298-4ba0-ac5a-ea3af6293936 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:31.939003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 53/97] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:31 2022] GET /identity/v3/role_assignments?user.id=1888f4b82c7940fdb9abfe0a57b404c5&scope.project.id=de138855cc8744659425af5f269b8fbc&role.id=20378c15bf524a4c960b49d2c53cf6b2 => generated 259 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:33.189322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e603505-1beb-4b3a-bccf-44b721377214 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:33.190085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e603505-1beb-4b3a-bccf-44b721377214 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:33.190085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e603505-1beb-4b3a-bccf-44b721377214 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:33.191311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 45/98] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:07:33.198146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72a84b-0ffe-41ae-90f8-8e25e3917b69 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:33.199245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72a84b-0ffe-41ae-90f8-8e25e3917b69 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:33.199722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72a84b-0ffe-41ae-90f8-8e25e3917b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:33.221472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0c72a84b-0ffe-41ae-90f8-8e25e3917b69 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:33.223115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0c72a84b-0ffe-41ae-90f8-8e25e3917b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:33.308207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 54/99] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:07:33.668075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815a5cb-c247-48ca-b53e-57c59254aadc None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:33.668338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815a5cb-c247-48ca-b53e-57c59254aadc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:33.668672 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815a5cb-c247-48ca-b53e-57c59254aadc None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:33.670026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 46/100] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:33.683872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d502741-2692-4ce4-b564-59b149894e8a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:33.684244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d502741-2692-4ce4-b564-59b149894e8a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:33.684454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d502741-2692-4ce4-b564-59b149894e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:33.711277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9d502741-2692-4ce4-b564-59b149894e8a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:33.712616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9d502741-2692-4ce4-b564-59b149894e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:33.745166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:33.811814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 55/101] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:33.888513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815a5cb-c247-48ca-b53e-57c59254aadc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:34.029795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815a5cb-c247-48ca-b53e-57c59254aadc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:34.070075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:34.076458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9bb86-41a8-4e3d-bc58-80db365a846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:34.077120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9bb86-41a8-4e3d-bc58-80db365a846d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:34.077571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9bb86-41a8-4e3d-bc58-80db365a846d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:34.078448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9bb86-41a8-4e3d-bc58-80db365a846d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:34.097373 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9bb86-41a8-4e3d-bc58-80db365a846d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d6d9bb86-41a8-4e3d-bc58-80db365a846d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:34.102617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:34.104314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:34.105822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 47/102] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:34.116758 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d502741-2692-4ce4-b564-59b149894e8a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:34.259970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d502741-2692-4ce4-b564-59b149894e8a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:34.304434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d5ac-086d-4871-9552-fc046546f4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:34.304964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d5ac-086d-4871-9552-fc046546f4fb None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:34.305412 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d5ac-086d-4871-9552-fc046546f4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:34.305695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f0d5ac-086d-4871-9552-fc046546f4fb None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:34.306405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0d5ac-086d-4871-9552-fc046546f4fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:34.306712 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0d5ac-086d-4871-9552-fc046546f4fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:34.308113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f0d5ac-086d-4871-9552-fc046546f4fb None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:34.317815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 56/103] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:07:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:34.319871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9bb86-41a8-4e3d-bc58-80db365a846d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:34.323941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d5ac-086d-4871-9552-fc046546f4fb None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:34.375699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:34.641314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f0d5ac-086d-4871-9552-fc046546f4fb None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:34.650170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9bb86-41a8-4e3d-bc58-80db365a846d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:34.736376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49510b-0815-4d9f-b28b-c1fbb2509142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:34.737386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49510b-0815-4d9f-b28b-c1fbb2509142 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:34.737690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49510b-0815-4d9f-b28b-c1fbb2509142 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:34.737975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49510b-0815-4d9f-b28b-c1fbb2509142 None admin] PATH_INFO: `/v3/users/e874b6c6bd824b4b875c5ce31404e4b9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:34.763655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49510b-0815-4d9f-b28b-c1fbb2509142 None admin] RBAC: Authorizing `identity:get_user(user_id=e874b6c6bd824b4b875c5ce31404e4b9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:34.765149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49510b-0815-4d9f-b28b-c1fbb2509142 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:34.767330 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 57/104] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:34 2022] GET /identity/v3/users/e874b6c6bd824b4b875c5ce31404e4b9 => generated 289 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:34.775901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49510b-0815-4d9f-b28b-c1fbb2509142 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:34.814443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffdec1-52a4-4f78-9bb3-62eb8a4cd3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:34.815536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffdec1-52a4-4f78-9bb3-62eb8a4cd3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:34.816028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffdec1-52a4-4f78-9bb3-62eb8a4cd3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:34.816515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffdec1-52a4-4f78-9bb3-62eb8a4cd3d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:34.823988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffdec1-52a4-4f78-9bb3-62eb8a4cd3d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:34.827049 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffdec1-52a4-4f78-9bb3-62eb8a4cd3d9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:34.830751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 48/105] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 512 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:34.841780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffdec1-52a4-4f78-9bb3-62eb8a4cd3d9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:35.002703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:35.069936 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:35.093193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffdec1-52a4-4f78-9bb3-62eb8a4cd3d9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:35.131541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49510b-0815-4d9f-b28b-c1fbb2509142 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:35.180922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8ecd1-ab85-43ce-9c49-f8a06bb4fa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:35.182459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8ecd1-ab85-43ce-9c49-f8a06bb4fa81 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:35.183177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8ecd1-ab85-43ce-9c49-f8a06bb4fa81 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:35.183907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8ecd1-ab85-43ce-9c49-f8a06bb4fa81 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:35.185499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8ecd1-ab85-43ce-9c49-f8a06bb4fa81 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:35.188705 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8ecd1-ab85-43ce-9c49-f8a06bb4fa81 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:35.237393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9167a78-9f7f-4060-a66d-b5a57c34470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:35.238491 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9167a78-9f7f-4060-a66d-b5a57c34470d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:35.240547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 49/106] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:07:34 2022] POST /identity/v3/projects => generated 302 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:07:35.241328 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9167a78-9f7f-4060-a66d-b5a57c34470d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:35.241807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9167a78-9f7f-4060-a66d-b5a57c34470d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:35.244026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671df411-0d07-412a-bab3-a0435635e3bc None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:35.244919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671df411-0d07-412a-bab3-a0435635e3bc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:35.245882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671df411-0d07-412a-bab3-a0435635e3bc None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:35.249017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 50/107] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:35.264772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa85e7e-ed0f-4ec6-804f-615a0888cea6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:35.265798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa85e7e-ed0f-4ec6-804f-615a0888cea6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:35.266677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa85e7e-ed0f-4ec6-804f-615a0888cea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:35.303340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5aa85e7e-ed0f-4ec6-804f-615a0888cea6 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:35.305785 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5aa85e7e-ed0f-4ec6-804f-615a0888cea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:35.476337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 51/108] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:07:35.494160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9167a78-9f7f-4060-a66d-b5a57c34470d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:35.497001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9167a78-9f7f-4060-a66d-b5a57c34470d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:35.663406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:35.704265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 58/109] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 929 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 16:07:35.713766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa85e7e-ed0f-4ec6-804f-615a0888cea6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:35.794258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:35.872325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa85e7e-ed0f-4ec6-804f-615a0888cea6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:35.926958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bc03f-6839-4794-9867-09976df124f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:35.927534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bc03f-6839-4794-9867-09976df124f4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:35.927826 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bc03f-6839-4794-9867-09976df124f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:35.928110 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bc03f-6839-4794-9867-09976df124f4 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:35.929398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bc03f-6839-4794-9867-09976df124f4 None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:35.930913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bc03f-6839-4794-9867-09976df124f4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:35.932899 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 52/110] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:35 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:35.942644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9167a78-9f7f-4060-a66d-b5a57c34470d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:36.015011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095a759-1aa7-4190-9674-33c9039b3733 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:36.015424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095a759-1aa7-4190-9674-33c9039b3733 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:36.016199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095a759-1aa7-4190-9674-33c9039b3733 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:36.017570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 53/111] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:07:36.023205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c0c61-b5c5-4674-8dac-85ae50db6a91 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:36.023543 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c0c61-b5c5-4674-8dac-85ae50db6a91 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:36.023778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c0c61-b5c5-4674-8dac-85ae50db6a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:36.069702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a57c0c61-b5c5-4674-8dac-85ae50db6a91 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:36.071694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a57c0c61-b5c5-4674-8dac-85ae50db6a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:36.316146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9167a78-9f7f-4060-a66d-b5a57c34470d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:36.326276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 54/112] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:07:36.329051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c0c61-b5c5-4674-8dac-85ae50db6a91 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:36.447028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:36.451907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9afb3b0-6539-4087-90c8-e466acf8c640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:36.454612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afb3b0-6539-4087-90c8-e466acf8c640 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:36.455504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afb3b0-6539-4087-90c8-e466acf8c640 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:36.456313 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afb3b0-6539-4087-90c8-e466acf8c640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:36.457970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afb3b0-6539-4087-90c8-e466acf8c640 None admin] RBAC: Adding query filter params (role.id=bbbfb342bd244610906d836359f61fcf, scope.project.id=832b79a184af40d7ad323b5e97816800, user.id=e874b6c6bd824b4b875c5ce31404e4b9) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:36.458848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afb3b0-6539-4087-90c8-e466acf8c640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:36.462701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9afb3b0-6539-4087-90c8-e466acf8c640 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:36.480694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 59/113] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:35 2022] GET /identity/v3/role_assignments?user.id=e874b6c6bd824b4b875c5ce31404e4b9&scope.project.id=832b79a184af40d7ad323b5e97816800&role.id=bbbfb342bd244610906d836359f61fcf => generated 259 bytes in 539 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:36.581176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c0c61-b5c5-4674-8dac-85ae50db6a91 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:36.592824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2134bf-9248-4b6a-ade3-c53ef83cdf3e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:36.593360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2134bf-9248-4b6a-ade3-c53ef83cdf3e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:36.594564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2134bf-9248-4b6a-ade3-c53ef83cdf3e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:36.597124 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 60/114] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:36.605556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1e3ba-4759-45b7-91be-2e87220eafa6 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:36.606132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1e3ba-4759-45b7-91be-2e87220eafa6 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:36.607247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1e3ba-4759-45b7-91be-2e87220eafa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:36.656988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d3e1e3ba-4759-45b7-91be-2e87220eafa6 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:36.658695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d3e1e3ba-4759-45b7-91be-2e87220eafa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:36.678738 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d20d7b-5560-403b-8d90-7534e511e47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:36.679411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d20d7b-5560-403b-8d90-7534e511e47d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:36.679697 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d20d7b-5560-403b-8d90-7534e511e47d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:36.679977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d20d7b-5560-403b-8d90-7534e511e47d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:36.712319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d20d7b-5560-403b-8d90-7534e511e47d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-23d20d7b-5560-403b-8d90-7534e511e47d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:36.717536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:36.718565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:36.719814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:36.719814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:36.719814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:36.719814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:36.719814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:36.719814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:36.719814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:36.719814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:36.719814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:36.719814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:36.719814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:07:36.719814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:36.730752 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 55/115] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:07:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:36.732697 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14fb5b-97c5-4bc3-aef8-421318ba3fa9 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:36.732994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14fb5b-97c5-4bc3-aef8-421318ba3fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:36.733277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14fb5b-97c5-4bc3-aef8-421318ba3fa9 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:36.734689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 56/116] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:36.736979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:36.743780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea3155-49d2-4cdb-9ed0-f1ce769264c4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:36.744083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea3155-49d2-4cdb-9ed0-f1ce769264c4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:36.744327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea3155-49d2-4cdb-9ed0-f1ce769264c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:36.790881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-baea3155-49d2-4cdb-9ed0-f1ce769264c4 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:36.792142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-baea3155-49d2-4cdb-9ed0-f1ce769264c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:36.804503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 61/117] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:36.805800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1e3ba-4759-45b7-91be-2e87220eafa6 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:36.887316 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 57/118] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:36.995570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1e3ba-4759-45b7-91be-2e87220eafa6 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:37.024506 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea3155-49d2-4cdb-9ed0-f1ce769264c4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:37.089419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117855f6-3da2-4270-9845-c980c686cb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:37.090731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117855f6-3da2-4270-9845-c980c686cb3a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:37.091250 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117855f6-3da2-4270-9845-c980c686cb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:37.091759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117855f6-3da2-4270-9845-c980c686cb3a None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:37.093270 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117855f6-3da2-4270-9845-c980c686cb3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:37.094464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117855f6-3da2-4270-9845-c980c686cb3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:37.097409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117855f6-3da2-4270-9845-c980c686cb3a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:37.107490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:37.127128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 62/119] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:07:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:37.133474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117855f6-3da2-4270-9845-c980c686cb3a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:37.242817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:37.298036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea3155-49d2-4cdb-9ed0-f1ce769264c4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:37.369640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed34cc6-44ee-41ac-aa3d-22baf661d944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:37.371160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed34cc6-44ee-41ac-aa3d-22baf661d944 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:37.371670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed34cc6-44ee-41ac-aa3d-22baf661d944 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:37.372144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed34cc6-44ee-41ac-aa3d-22baf661d944 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:37.386765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117855f6-3da2-4270-9845-c980c686cb3a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:37.393576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed34cc6-44ee-41ac-aa3d-22baf661d944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-fed34cc6-44ee-41ac-aa3d-22baf661d944 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:37.403403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:37.404286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:37.405044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:37.405044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:37.405044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:37.405044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:37.405044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:37.405044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:37.405044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:37.405044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:37.405044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:37.405044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:37.405044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:37.405044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:37.406223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 58/120] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:07:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:07:37.407325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed34cc6-44ee-41ac-aa3d-22baf661d944 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:37.458895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a35b9b-90be-4dd0-87f5-325902772e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:37.460314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a35b9b-90be-4dd0-87f5-325902772e5b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:37.461118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a35b9b-90be-4dd0-87f5-325902772e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:37.461583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a35b9b-90be-4dd0-87f5-325902772e5b None admin] PATH_INFO: `/v3/users/1888f4b82c7940fdb9abfe0a57b404c5` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:37.463482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a35b9b-90be-4dd0-87f5-325902772e5b None admin] RBAC: Authorizing `identity:get_user(user_id=1888f4b82c7940fdb9abfe0a57b404c5)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:37.466213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a35b9b-90be-4dd0-87f5-325902772e5b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:37.470731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 63/121] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:37 2022] GET /identity/v3/users/1888f4b82c7940fdb9abfe0a57b404c5 => generated 275 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:37.472289 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a35b9b-90be-4dd0-87f5-325902772e5b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:37.598719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:37.629402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed34cc6-44ee-41ac-aa3d-22baf661d944 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:37.706630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a35b9b-90be-4dd0-87f5-325902772e5b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:37.707597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071f6bd-1ea2-466f-8d74-2db7737ad79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:37.708676 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071f6bd-1ea2-466f-8d74-2db7737ad79b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:37.709172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071f6bd-1ea2-466f-8d74-2db7737ad79b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:37.709688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071f6bd-1ea2-466f-8d74-2db7737ad79b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:37.712047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:37.733275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071f6bd-1ea2-466f-8d74-2db7737ad79b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0071f6bd-1ea2-466f-8d74-2db7737ad79b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:37.745010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:37.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:37.746784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:37.746784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:37.746784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:37.746784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:37.746784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:37.746784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:37.746784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:37.746784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:37.746784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:37.746784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:37.746784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:37.746784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:37.747986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 59/122] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:37.752451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071f6bd-1ea2-466f-8d74-2db7737ad79b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:37.784818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971f0b5-ce69-43ce-a1f6-53dbc31a18b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:37.785919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f0b5-ce69-43ce-a1f6-53dbc31a18b9 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:37.787068 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f0b5-ce69-43ce-a1f6-53dbc31a18b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:37.787556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c971f0b5-ce69-43ce-a1f6-53dbc31a18b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:37.811007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c971f0b5-ce69-43ce-a1f6-53dbc31a18b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-c971f0b5-ce69-43ce-a1f6-53dbc31a18b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:37.824074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:37.825868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:07:37.827405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 64/123] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:37.828910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971f0b5-ce69-43ce-a1f6-53dbc31a18b9 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:37.979247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071f6bd-1ea2-466f-8d74-2db7737ad79b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:38.052104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc3c4f-2ca6-42c1-8642-e1ead12684d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:38.053328 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc3c4f-2ca6-42c1-8642-e1ead12684d8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:38.054164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc3c4f-2ca6-42c1-8642-e1ead12684d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:38.054743 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc3c4f-2ca6-42c1-8642-e1ead12684d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:38.055993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c971f0b5-ce69-43ce-a1f6-53dbc31a18b9 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:38.056158 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc3c4f-2ca6-42c1-8642-e1ead12684d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:38.056785 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc3c4f-2ca6-42c1-8642-e1ead12684d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:38.059868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc3c4f-2ca6-42c1-8642-e1ead12684d8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:38.081578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 60/124] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:38.083631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc3c4f-2ca6-42c1-8642-e1ead12684d8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:38.110780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:38.133705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc819bd5-6211-4182-a6bf-0aa1cf4b8a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:38.134918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc819bd5-6211-4182-a6bf-0aa1cf4b8a9f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:38.136178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc819bd5-6211-4182-a6bf-0aa1cf4b8a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:38.136744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc819bd5-6211-4182-a6bf-0aa1cf4b8a9f None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:38.139481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc819bd5-6211-4182-a6bf-0aa1cf4b8a9f None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:38.142922 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc819bd5-6211-4182-a6bf-0aa1cf4b8a9f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:38.147445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 65/125] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:37 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:38.148967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc819bd5-6211-4182-a6bf-0aa1cf4b8a9f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:38.216650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:38.313407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc3c4f-2ca6-42c1-8642-e1ead12684d8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:38.373447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc819bd5-6211-4182-a6bf-0aa1cf4b8a9f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:38.385635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9e3b1-ad31-4392-af10-70b7421d6471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:38.387067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9e3b1-ad31-4392-af10-70b7421d6471 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:38.387750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9e3b1-ad31-4392-af10-70b7421d6471 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:38.388443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9e3b1-ad31-4392-af10-70b7421d6471 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:38.389860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9e3b1-ad31-4392-af10-70b7421d6471 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:38.390738 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9e3b1-ad31-4392-af10-70b7421d6471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:38.393652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a9e3b1-ad31-4392-af10-70b7421d6471 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:38.412059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 61/126] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:07:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:38.413888 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9e3b1-ad31-4392-af10-70b7421d6471 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:38.448602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ee302-dc28-48d5-9215-7aa49ac22619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:38.449652 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ee302-dc28-48d5-9215-7aa49ac22619 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:38.450525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ee302-dc28-48d5-9215-7aa49ac22619 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:38.451035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ee302-dc28-48d5-9215-7aa49ac22619 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:38.452365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ee302-dc28-48d5-9215-7aa49ac22619 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:38.453347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ee302-dc28-48d5-9215-7aa49ac22619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:38.456987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674ee302-dc28-48d5-9215-7aa49ac22619 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:38.476460 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 66/127] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:07:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:38.477731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ee302-dc28-48d5-9215-7aa49ac22619 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:38.564541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:38.620271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9e3b1-ad31-4392-af10-70b7421d6471 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:38.645295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:38.657311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ee302-dc28-48d5-9215-7aa49ac22619 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:38.682778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37fd76-d36b-4bd8-a6ed-599c9cbdedc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:38.684219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37fd76-d36b-4bd8-a6ed-599c9cbdedc9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:38.684981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37fd76-d36b-4bd8-a6ed-599c9cbdedc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:38.685706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37fd76-d36b-4bd8-a6ed-599c9cbdedc9 None admin] PATH_INFO: `/v3/users/b9a31b5d481f471a90714fa648bfed35` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:38.688451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37fd76-d36b-4bd8-a6ed-599c9cbdedc9 None admin] RBAC: Authorizing `identity:get_user(user_id=b9a31b5d481f471a90714fa648bfed35)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:38.691292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b37fd76-d36b-4bd8-a6ed-599c9cbdedc9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:38.694037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 62/128] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:38 2022] GET /identity/v3/users/b9a31b5d481f471a90714fa648bfed35 => generated 247 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:38.695409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37fd76-d36b-4bd8-a6ed-599c9cbdedc9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:38.728766 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b9fc9f-2655-4fd1-8a1b-b4177db5c6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:38.730769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9fc9f-2655-4fd1-8a1b-b4177db5c6c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:38.731231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9fc9f-2655-4fd1-8a1b-b4177db5c6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:38.731690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b9fc9f-2655-4fd1-8a1b-b4177db5c6c0 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800/users/1888f4b82c7940fdb9abfe0a57b404c5/roles/aa320f864d914f65bebe5463994ff536` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:38.736250 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9fc9f-2655-4fd1-8a1b-b4177db5c6c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=832b79a184af40d7ad323b5e97816800, user_id=1888f4b82c7940fdb9abfe0a57b404c5, role_id=aa320f864d914f65bebe5463994ff536)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:38.738722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b9fc9f-2655-4fd1-8a1b-b4177db5c6c0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:38.758149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 67/129] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:07:38 2022] PUT /identity/v3/projects/832b79a184af40d7ad323b5e97816800/users/1888f4b82c7940fdb9abfe0a57b404c5/roles/aa320f864d914f65bebe5463994ff536 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:07:38.759928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea4fb4-441c-436b-8966-eb7040ff3b17 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:38.760440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea4fb4-441c-436b-8966-eb7040ff3b17 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:38.760951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea4fb4-441c-436b-8966-eb7040ff3b17 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:38.774078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 68/130] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:38.774078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea4fb4-441c-436b-8966-eb7040ff3b17 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:38.910799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37fd76-d36b-4bd8-a6ed-599c9cbdedc9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:38.978540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c459a56-5707-4cbe-9bd2-54b97d68110b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:38.979221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c459a56-5707-4cbe-9bd2-54b97d68110b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:38.979511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c459a56-5707-4cbe-9bd2-54b97d68110b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:38.979794 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c459a56-5707-4cbe-9bd2-54b97d68110b None admin] PATH_INFO: `/v3/users/1888f4b82c7940fdb9abfe0a57b404c5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:38.981166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c459a56-5707-4cbe-9bd2-54b97d68110b None admin] RBAC: Authorizing `identity:get_user(user_id=1888f4b82c7940fdb9abfe0a57b404c5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:38.982764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c459a56-5707-4cbe-9bd2-54b97d68110b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:38.984609 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 63/131] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:38 2022] GET /identity/v3/users/1888f4b82c7940fdb9abfe0a57b404c5 => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:38.985490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c459a56-5707-4cbe-9bd2-54b97d68110b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:39.005218 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea4fb4-441c-436b-8966-eb7040ff3b17 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:39.024449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:39.050601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efb45c-71e2-402f-ac1c-0ca891ef8134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:39.051269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efb45c-71e2-402f-ac1c-0ca891ef8134 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:39.051557 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efb45c-71e2-402f-ac1c-0ca891ef8134 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:39.051823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efb45c-71e2-402f-ac1c-0ca891ef8134 None admin] PATH_INFO: `/v3/users/1888f4b82c7940fdb9abfe0a57b404c5` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:39.053102 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efb45c-71e2-402f-ac1c-0ca891ef8134 None admin] RBAC: Authorizing `identity:get_user(user_id=1888f4b82c7940fdb9abfe0a57b404c5)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:39.055804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efb45c-71e2-402f-ac1c-0ca891ef8134 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:39.057844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 69/132] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:38 2022] GET /identity/v3/users/1888f4b82c7940fdb9abfe0a57b404c5 => generated 275 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:39.058861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e62268-524f-41a8-8024-ad827a22e020 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:39.059149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e62268-524f-41a8-8024-ad827a22e020 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:39.059436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e62268-524f-41a8-8024-ad827a22e020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:39.082910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-09e62268-524f-41a8-8024-ad827a22e020 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:39.084524 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-09e62268-524f-41a8-8024-ad827a22e020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:39.110976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:39.160431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c459a56-5707-4cbe-9bd2-54b97d68110b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:39.182197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 70/133] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:39.182197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e62268-524f-41a8-8024-ad827a22e020 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:39.208294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27d556-42ba-4e97-9404-2225d7e449e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:39.209130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27d556-42ba-4e97-9404-2225d7e449e8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:39.209674 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27d556-42ba-4e97-9404-2225d7e449e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:39.210290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27d556-42ba-4e97-9404-2225d7e449e8 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:39.212448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27d556-42ba-4e97-9404-2225d7e449e8 None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:39.215050 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27d556-42ba-4e97-9404-2225d7e449e8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:39.222111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 64/134] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:38 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:39.222111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27d556-42ba-4e97-9404-2225d7e449e8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:39.345438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:39.351331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e62268-524f-41a8-8024-ad827a22e020 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:39.374010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27d556-42ba-4e97-9404-2225d7e449e8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:39.417463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd8976-6b22-4f56-b2f2-1f00c247952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:39.418426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd8976-6b22-4f56-b2f2-1f00c247952d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:39.418855 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd8976-6b22-4f56-b2f2-1f00c247952d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:39.419271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd8976-6b22-4f56-b2f2-1f00c247952d None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:39.420593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd8976-6b22-4f56-b2f2-1f00c247952d None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:39.420837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5a549-c6ee-427d-b582-9b9b2f91ffef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:39.422664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5a549-c6ee-427d-b582-9b9b2f91ffef None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:39.423138 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd8976-6b22-4f56-b2f2-1f00c247952d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:39.423400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5a549-c6ee-427d-b582-9b9b2f91ffef None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:39.423994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5a549-c6ee-427d-b582-9b9b2f91ffef None admin] PATH_INFO: `/v3/projects/de138855cc8744659425af5f269b8fbc` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:39.425303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 65/135] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:39 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:39.426324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5a549-c6ee-427d-b582-9b9b2f91ffef None admin] RBAC: Authorizing `identity:get_project(project_id=de138855cc8744659425af5f269b8fbc)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:39.427077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7b8ac-b643-4252-8c8a-fa126881db18 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:39.427507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7b8ac-b643-4252-8c8a-fa126881db18 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:39.427926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7b8ac-b643-4252-8c8a-fa126881db18 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:39.429340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 66/136] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:39.429728 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d5a549-c6ee-427d-b582-9b9b2f91ffef None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:39.430458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7b8ac-b643-4252-8c8a-fa126881db18 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:39.432919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 71/137] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:39 2022] GET /identity/v3/projects/de138855cc8744659425af5f269b8fbc => generated 312 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:39.437963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0341d0-2af7-4d80-a7cb-27f2643c4c9a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:39.438732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0341d0-2af7-4d80-a7cb-27f2643c4c9a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:39.439340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0341d0-2af7-4d80-a7cb-27f2643c4c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:39.479325 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ad0341d0-2af7-4d80-a7cb-27f2643c4c9a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:39.480579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ad0341d0-2af7-4d80-a7cb-27f2643c4c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:39.504984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:39.581211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 72/138] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:07:39.583325 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0341d0-2af7-4d80-a7cb-27f2643c4c9a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:39.602878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7b8ac-b643-4252-8c8a-fa126881db18 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:39.695574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539d972-bfec-43b8-9bbe-41607654e968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:39.696528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539d972-bfec-43b8-9bbe-41607654e968 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:39.696528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539d972-bfec-43b8-9bbe-41607654e968 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:39.696774 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539d972-bfec-43b8-9bbe-41607654e968 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800/users/b9a31b5d481f471a90714fa648bfed35/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:39.704220 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539d972-bfec-43b8-9bbe-41607654e968 None admin] RBAC: Authorizing `identity:create_grant(project_id=832b79a184af40d7ad323b5e97816800, user_id=b9a31b5d481f471a90714fa648bfed35, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:39.705716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539d972-bfec-43b8-9bbe-41607654e968 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:39.733276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 67/139] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:07:39 2022] PUT /identity/v3/projects/832b79a184af40d7ad323b5e97816800/users/b9a31b5d481f471a90714fa648bfed35/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:07:39.739215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539d972-bfec-43b8-9bbe-41607654e968 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:39.750515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:39.768200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0341d0-2af7-4d80-a7cb-27f2643c4c9a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:39.846405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f0b87-6444-4336-bd1d-ff7e815d8fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:39.847162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f0b87-6444-4336-bd1d-ff7e815d8fbe None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:39.847569 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f0b87-6444-4336-bd1d-ff7e815d8fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:39.847921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f0b87-6444-4336-bd1d-ff7e815d8fbe None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800/users/1888f4b82c7940fdb9abfe0a57b404c5/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:39.854093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f0b87-6444-4336-bd1d-ff7e815d8fbe None admin] RBAC: Authorizing `identity:create_grant(project_id=832b79a184af40d7ad323b5e97816800, user_id=1888f4b82c7940fdb9abfe0a57b404c5, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:39.854093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f0b87-6444-4336-bd1d-ff7e815d8fbe None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:39.885647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 73/140] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:07:39 2022] PUT /identity/v3/projects/832b79a184af40d7ad323b5e97816800/users/1888f4b82c7940fdb9abfe0a57b404c5/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:07:39.886701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f0b87-6444-4336-bd1d-ff7e815d8fbe None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:39.926672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:39.994270 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539d972-bfec-43b8-9bbe-41607654e968 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:40.076845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a824e-196a-4e14-8df0-13380a41248e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:40.077575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a824e-196a-4e14-8df0-13380a41248e None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:40.078146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a824e-196a-4e14-8df0-13380a41248e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:40.078436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a824e-196a-4e14-8df0-13380a41248e None admin] PATH_INFO: `/v3/projects/de138855cc8744659425af5f269b8fbc/users/1888f4b82c7940fdb9abfe0a57b404c5/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:40.083162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a824e-196a-4e14-8df0-13380a41248e None admin] RBAC: Authorizing `identity:create_grant(project_id=de138855cc8744659425af5f269b8fbc, user_id=1888f4b82c7940fdb9abfe0a57b404c5, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:40.084884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a824e-196a-4e14-8df0-13380a41248e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:40.108165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 68/141] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:07:39 2022] PUT /identity/v3/projects/de138855cc8744659425af5f269b8fbc/users/1888f4b82c7940fdb9abfe0a57b404c5/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:07:40.109014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a824e-196a-4e14-8df0-13380a41248e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:40.127364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f0b87-6444-4336-bd1d-ff7e815d8fbe None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:40.177813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9fc8c-f8cd-449e-9712-d2ae26b1dd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:40.178528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9fc8c-f8cd-449e-9712-d2ae26b1dd45 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:40.178867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9fc8c-f8cd-449e-9712-d2ae26b1dd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:40.179190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9fc8c-f8cd-449e-9712-d2ae26b1dd45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:40.180483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9fc8c-f8cd-449e-9712-d2ae26b1dd45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:40.182036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9fc8c-f8cd-449e-9712-d2ae26b1dd45 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:40.184825 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 74/142] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:40.190979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9fc8c-f8cd-449e-9712-d2ae26b1dd45 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:40.199676 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:40.250217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a824e-196a-4e14-8df0-13380a41248e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:40.294086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca7ea1-2419-4835-b0d3-e3509299dd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:40.294778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca7ea1-2419-4835-b0d3-e3509299dd32 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:40.295070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca7ea1-2419-4835-b0d3-e3509299dd32 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:40.295358 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca7ea1-2419-4835-b0d3-e3509299dd32 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:40.309970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca7ea1-2419-4835-b0d3-e3509299dd32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-e8ca7ea1-2419-4835-b0d3-e3509299dd32 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:40.316994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:40.320300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:40.323580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 69/143] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:40.331838 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca7ea1-2419-4835-b0d3-e3509299dd32 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:40.341766 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:40.399107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9fc8c-f8cd-449e-9712-d2ae26b1dd45 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:40.482930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce294a99-57b1-4643-973d-08efb437b559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:40.483587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce294a99-57b1-4643-973d-08efb437b559 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:40.484438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce294a99-57b1-4643-973d-08efb437b559 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:40.484719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce294a99-57b1-4643-973d-08efb437b559 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:40.485433 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce294a99-57b1-4643-973d-08efb437b559 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:40.487046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce294a99-57b1-4643-973d-08efb437b559 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:40.501366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca7ea1-2419-4835-b0d3-e3509299dd32 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:40.526196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 75/144] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:07:40 2022] POST /identity/v3/users => generated 283 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:07:40.541770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:40.547607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292df96-b1f2-45d6-ada6-77f09d922776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:40.548599 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292df96-b1f2-45d6-ada6-77f09d922776 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:40.549067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292df96-b1f2-45d6-ada6-77f09d922776 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:40.549530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292df96-b1f2-45d6-ada6-77f09d922776 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:40.550742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292df96-b1f2-45d6-ada6-77f09d922776 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:40.551268 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292df96-b1f2-45d6-ada6-77f09d922776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:40.552957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292df96-b1f2-45d6-ada6-77f09d922776 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:40.564049 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 70/145] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:07:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:40.570439 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce294a99-57b1-4643-973d-08efb437b559 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:40.772264 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce294a99-57b1-4643-973d-08efb437b559 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:40.860800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d03a4-47a5-4206-859b-6be2f53f5488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:40.862346 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d03a4-47a5-4206-859b-6be2f53f5488 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:40.863115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d03a4-47a5-4206-859b-6be2f53f5488 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:40.863613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255d03a4-47a5-4206-859b-6be2f53f5488 None admin] PATH_INFO: `/v3/users/e874b6c6bd824b4b875c5ce31404e4b9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:40.867979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d03a4-47a5-4206-859b-6be2f53f5488 None admin] RBAC: Authorizing `identity:get_user(user_id=e874b6c6bd824b4b875c5ce31404e4b9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:40.870901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255d03a4-47a5-4206-859b-6be2f53f5488 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:40.874277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 76/146] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:40 2022] GET /identity/v3/users/e874b6c6bd824b4b875c5ce31404e4b9 => generated 289 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:40.883367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292df96-b1f2-45d6-ada6-77f09d922776 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:41.043861 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292df96-b1f2-45d6-ada6-77f09d922776 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:41.086081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edcde8-beb9-44ce-adc4-2d69f029a12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:41.086774 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edcde8-beb9-44ce-adc4-2d69f029a12e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:41.087167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edcde8-beb9-44ce-adc4-2d69f029a12e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:41.087449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edcde8-beb9-44ce-adc4-2d69f029a12e None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:41.088805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edcde8-beb9-44ce-adc4-2d69f029a12e None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:41.090400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edcde8-beb9-44ce-adc4-2d69f029a12e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:41.092490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 71/147] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:40 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:41.099088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d03a4-47a5-4206-859b-6be2f53f5488 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:41.102179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:41.227247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255d03a4-47a5-4206-859b-6be2f53f5488 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:41.277776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332c80f-2ddd-4ae9-b4eb-34bb12ab8481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:41.278607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332c80f-2ddd-4ae9-b4eb-34bb12ab8481 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:41.278893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332c80f-2ddd-4ae9-b4eb-34bb12ab8481 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:41.279173 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332c80f-2ddd-4ae9-b4eb-34bb12ab8481 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800/users/e874b6c6bd824b4b875c5ce31404e4b9/roles/bbbfb342bd244610906d836359f61fcf` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:41.281405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0332c80f-2ddd-4ae9-b4eb-34bb12ab8481 None admin] RBAC: Authorizing `identity:create_grant(project_id=832b79a184af40d7ad323b5e97816800, user_id=e874b6c6bd824b4b875c5ce31404e4b9, role_id=bbbfb342bd244610906d836359f61fcf)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:41.283039 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0332c80f-2ddd-4ae9-b4eb-34bb12ab8481 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:41.309187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 77/148] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:07:41 2022] PUT /identity/v3/projects/832b79a184af40d7ad323b5e97816800/users/e874b6c6bd824b4b875c5ce31404e4b9/roles/bbbfb342bd244610906d836359f61fcf => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:07:42.656326 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af336e6d-3892-478c-ae4a-0a174871fe70 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:42.656858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af336e6d-3892-478c-ae4a-0a174871fe70 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:42.656932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af336e6d-3892-478c-ae4a-0a174871fe70 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:42.658666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 72/149] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:42.667011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ba750-9bbe-45de-b3ba-847d851187ef None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:42.667506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ba750-9bbe-45de-b3ba-847d851187ef None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:42.667577 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ba750-9bbe-45de-b3ba-847d851187ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:42.695208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-686ba750-9bbe-45de-b3ba-847d851187ef None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:42.696450 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-686ba750-9bbe-45de-b3ba-847d851187ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:42.776847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:42.791828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 78/150] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:42.959819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc9070-e87b-46af-bc43-bec9b611cb03 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:42.960207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc9070-e87b-46af-bc43-bec9b611cb03 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:42.960384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc9070-e87b-46af-bc43-bec9b611cb03 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:42.962104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 73/151] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:42.970598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a297f0e-b639-48cb-99ac-756278fabf4c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:42.970895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a297f0e-b639-48cb-99ac-756278fabf4c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:42.971149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a297f0e-b639-48cb-99ac-756278fabf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:42.996513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3a297f0e-b639-48cb-99ac-756278fabf4c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:42.997769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3a297f0e-b639-48cb-99ac-756278fabf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:43.098634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 79/152] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:43.676619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc9070-e87b-46af-bc43-bec9b611cb03 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:43.763660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:43.831864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc9070-e87b-46af-bc43-bec9b611cb03 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:43.883561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a297f0e-b639-48cb-99ac-756278fabf4c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:43.904731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95c19c-46df-44c8-9ede-cb37358c8852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:43.905451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95c19c-46df-44c8-9ede-cb37358c8852 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:43.906321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95c19c-46df-44c8-9ede-cb37358c8852 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:43.906602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95c19c-46df-44c8-9ede-cb37358c8852 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:43.944698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95c19c-46df-44c8-9ede-cb37358c8852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-fc95c19c-46df-44c8-9ede-cb37358c8852 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:43.949922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:43.950934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:43.951683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:43.951683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:43.951683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:43.951683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:43.951683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:43.951683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:43.951683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:43.951683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:43.951683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:43.951683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:43.951683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:43.951683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:43.955773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 74/153] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:07:43.961910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95c19c-46df-44c8-9ede-cb37358c8852 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:44.109486 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95c19c-46df-44c8-9ede-cb37358c8852 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:44.127375 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:44.215567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a297f0e-b639-48cb-99ac-756278fabf4c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:44.219257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443d977-e1c3-44dc-8f1b-ea8fc0aa1bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:44.219902 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443d977-e1c3-44dc-8f1b-ea8fc0aa1bce None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:44.220193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443d977-e1c3-44dc-8f1b-ea8fc0aa1bce None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:44.220453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443d977-e1c3-44dc-8f1b-ea8fc0aa1bce None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:44.221152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443d977-e1c3-44dc-8f1b-ea8fc0aa1bce None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:44.221456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443d977-e1c3-44dc-8f1b-ea8fc0aa1bce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:44.222898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443d977-e1c3-44dc-8f1b-ea8fc0aa1bce None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:44.232525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 75/154] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:07:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:44.233688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4f9c5-e51b-4e6d-90db-9006df0229b1 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:44.233979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4f9c5-e51b-4e6d-90db-9006df0229b1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:44.242471 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4f9c5-e51b-4e6d-90db-9006df0229b1 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:44.244154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 76/155] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:44 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:44.244707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4f9c5-e51b-4e6d-90db-9006df0229b1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:44.267010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:44.294269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18702b1f-a52a-4a63-bb40-25e8aea8b024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:44.294930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18702b1f-a52a-4a63-bb40-25e8aea8b024 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:44.295209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18702b1f-a52a-4a63-bb40-25e8aea8b024 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:44.295492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18702b1f-a52a-4a63-bb40-25e8aea8b024 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:44.334034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18702b1f-a52a-4a63-bb40-25e8aea8b024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-18702b1f-a52a-4a63-bb40-25e8aea8b024 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:44.348002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:44.349829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:44.351407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:44.351407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:44.351407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:44.351407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:44.351407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:44.351407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:44.351407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:44.351407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:44.351407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:44.351407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:44.351407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:07:44.351407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:07:44.355852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 80/156] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:07:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 473 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:44.362792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb127e0-dbea-421e-80d0-def3312348e1 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:44.363731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb127e0-dbea-421e-80d0-def3312348e1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:44.364867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb127e0-dbea-421e-80d0-def3312348e1 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:44.369348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 81/157] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:44 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:44.372037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e12303-0cf5-4b45-9aaa-b75498d9151e None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:44.372764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e12303-0cf5-4b45-9aaa-b75498d9151e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:44.373473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e12303-0cf5-4b45-9aaa-b75498d9151e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:44.411801 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-82e12303-0cf5-4b45-9aaa-b75498d9151e None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:44.414392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-82e12303-0cf5-4b45-9aaa-b75498d9151e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:44.487922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4f9c5-e51b-4e6d-90db-9006df0229b1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:44.550370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 82/158] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:44.552776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c71a6-3d60-4293-8b62-29b4a42d3962 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:44.553278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c71a6-3d60-4293-8b62-29b4a42d3962 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:44.553770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49c71a6-3d60-4293-8b62-29b4a42d3962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:44.579777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8a969-3fbd-4980-9dc4-9c0b18515dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:44.580443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8a969-3fbd-4980-9dc4-9c0b18515dad None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:44.580715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8a969-3fbd-4980-9dc4-9c0b18515dad None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:44.580993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8a969-3fbd-4980-9dc4-9c0b18515dad None admin] PATH_INFO: `/v3/users/1888f4b82c7940fdb9abfe0a57b404c5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:44.585914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a49c71a6-3d60-4293-8b62-29b4a42d3962 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:44.586793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8a969-3fbd-4980-9dc4-9c0b18515dad None admin] RBAC: Authorizing `identity:get_user(user_id=1888f4b82c7940fdb9abfe0a57b404c5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:44.587991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a49c71a6-3d60-4293-8b62-29b4a42d3962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:44.588287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8a969-3fbd-4980-9dc4-9c0b18515dad None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:44.589959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 77/159] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:44 2022] GET /identity/v3/users/1888f4b82c7940fdb9abfe0a57b404c5 => generated 275 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:44.590852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8a969-3fbd-4980-9dc4-9c0b18515dad None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:44.653226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:44.768888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 83/160] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:44.770242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c71a6-3d60-4293-8b62-29b4a42d3962 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:44.793746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8a969-3fbd-4980-9dc4-9c0b18515dad None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:44.866213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:44.896274 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946ceb0-de5c-4965-804e-87c132d261bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:44.897317 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946ceb0-de5c-4965-804e-87c132d261bc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:44.903366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946ceb0-de5c-4965-804e-87c132d261bc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:44.903861 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946ceb0-de5c-4965-804e-87c132d261bc None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:44.905164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946ceb0-de5c-4965-804e-87c132d261bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:44.905793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946ceb0-de5c-4965-804e-87c132d261bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:44.909106 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3946ceb0-de5c-4965-804e-87c132d261bc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:44.941292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 78/161] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:07:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:44.948165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946ceb0-de5c-4965-804e-87c132d261bc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:45.085720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49c71a6-3d60-4293-8b62-29b4a42d3962 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:45.166565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d14e4-94e1-41db-9b35-784946c6c7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:45.167366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d14e4-94e1-41db-9b35-784946c6c7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:45.167654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d14e4-94e1-41db-9b35-784946c6c7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:45.167932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d14e4-94e1-41db-9b35-784946c6c7a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:45.309125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946ceb0-de5c-4965-804e-87c132d261bc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:45.352130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:45.398672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d14e4-94e1-41db-9b35-784946c6c7a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:45.400161 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d14e4-94e1-41db-9b35-784946c6c7a1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:45.470131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:45.485635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d050c-ba64-4cf6-9772-c20195aeeb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:45.494085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d050c-ba64-4cf6-9772-c20195aeeb90 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:45.494085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d050c-ba64-4cf6-9772-c20195aeeb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:45.494085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d050c-ba64-4cf6-9772-c20195aeeb90 None admin] PATH_INFO: `/v3/users/1888f4b82c7940fdb9abfe0a57b404c5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:45.494948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d050c-ba64-4cf6-9772-c20195aeeb90 None admin] RBAC: Authorizing `identity:get_user(user_id=1888f4b82c7940fdb9abfe0a57b404c5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:45.496526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d050c-ba64-4cf6-9772-c20195aeeb90 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:45.498298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 79/162] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:44 2022] GET /identity/v3/users/1888f4b82c7940fdb9abfe0a57b404c5 => generated 275 bytes in 551 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:45.500634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d050c-ba64-4cf6-9772-c20195aeeb90 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:45.595957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 84/163] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 826 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:07:45.596667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d14e4-94e1-41db-9b35-784946c6c7a1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:45.825559 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d050c-ba64-4cf6-9772-c20195aeeb90 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:45.869870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d14e4-94e1-41db-9b35-784946c6c7a1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:45.998797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a74e02-e531-4914-a73f-8b02c3596791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:46.001952 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a74e02-e531-4914-a73f-8b02c3596791 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.002474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a74e02-e531-4914-a73f-8b02c3596791 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.002957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a74e02-e531-4914-a73f-8b02c3596791 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.032159 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793e0c4-f3b1-424d-80b3-f9d4b5dce36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:46.032823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793e0c4-f3b1-424d-80b3-f9d4b5dce36f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.033113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793e0c4-f3b1-424d-80b3-f9d4b5dce36f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.033393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793e0c4-f3b1-424d-80b3-f9d4b5dce36f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.050078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a74e02-e531-4914-a73f-8b02c3596791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-62a74e02-e531-4914-a73f-8b02c3596791 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:46.059036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:46.059956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:46.060724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:46.060724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:46.060724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:46.060724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:46.060724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:46.060724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:46.060724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:46.060724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:46.060724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:46.060724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:46.060724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:46.060724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:46.065062 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:46.067155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 80/164] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:07:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 568 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:46.069034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a74e02-e531-4914-a73f-8b02c3596791 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:46.136068 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:46.203249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793e0c4-f3b1-424d-80b3-f9d4b5dce36f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:46.204764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793e0c4-f3b1-424d-80b3-f9d4b5dce36f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:46.272034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a74e02-e531-4914-a73f-8b02c3596791 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:46.349502 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca5e47-9716-4daa-ad12-c58f9bab8ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:46.351251 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca5e47-9716-4daa-ad12-c58f9bab8ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.351769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca5e47-9716-4daa-ad12-c58f9bab8ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.352296 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca5e47-9716-4daa-ad12-c58f9bab8ed6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.375955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca5e47-9716-4daa-ad12-c58f9bab8ed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:46.384617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 85/165] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 788 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:07:46.385737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793e0c4-f3b1-424d-80b3-f9d4b5dce36f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b1ca5e47-9716-4daa-ad12-c58f9bab8ed6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:46.386205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:46.387052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:46.387827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:46.387827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:46.387827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:46.387827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:46.387827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:46.387827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:46.387827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:46.387827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:46.387827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:46.387827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:46.387827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:46.387827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:46.388415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 81/166] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:46.389670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca5e47-9716-4daa-ad12-c58f9bab8ed6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:46.443234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:46.515312 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793e0c4-f3b1-424d-80b3-f9d4b5dce36f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:46.535220 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:46.543580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca5e47-9716-4daa-ad12-c58f9bab8ed6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:46.556649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bb765-222a-4ac0-8275-f0c1278ae3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:46.557419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bb765-222a-4ac0-8275-f0c1278ae3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.557778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bb765-222a-4ac0-8275-f0c1278ae3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.558254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bb765-222a-4ac0-8275-f0c1278ae3f4 None admin] PATH_INFO: `/v3/projects/de138855cc8744659425af5f269b8fbc` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.559763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bb765-222a-4ac0-8275-f0c1278ae3f4 None admin] RBAC: Authorizing `identity:get_project(project_id=de138855cc8744659425af5f269b8fbc)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:46.561985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bb765-222a-4ac0-8275-f0c1278ae3f4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:46.563721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 86/167] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:46 2022] GET /identity/v3/projects/de138855cc8744659425af5f269b8fbc => generated 312 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:46.565040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc12cc1-823b-4c6e-a35b-daffca5d3f4c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.565323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc12cc1-823b-4c6e-a35b-daffca5d3f4c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.565591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc12cc1-823b-4c6e-a35b-daffca5d3f4c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.566853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 87/168] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:46.568019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64436786-96d3-439f-bb66-1d6a48cd681d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.568301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64436786-96d3-439f-bb66-1d6a48cd681d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.568714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64436786-96d3-439f-bb66-1d6a48cd681d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.570355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 88/169] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:46.571389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ca720-346c-412e-8d01-a538238a1648 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.571730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ca720-346c-412e-8d01-a538238a1648 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.572003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ca720-346c-412e-8d01-a538238a1648 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.578121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 89/170] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:46.578121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ca720-346c-412e-8d01-a538238a1648 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:46.593974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b57e6-50e5-45a0-9436-140bc1b6c78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:46.594990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b57e6-50e5-45a0-9436-140bc1b6c78d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.595392 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b57e6-50e5-45a0-9436-140bc1b6c78d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.595807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b57e6-50e5-45a0-9436-140bc1b6c78d None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.596925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b57e6-50e5-45a0-9436-140bc1b6c78d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:46.597411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b57e6-50e5-45a0-9436-140bc1b6c78d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:46.599776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b57e6-50e5-45a0-9436-140bc1b6c78d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:46.610333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 82/171] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:46.611217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b57e6-50e5-45a0-9436-140bc1b6c78d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:46.689421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ca720-346c-412e-8d01-a538238a1648 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:46.701909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:46.717351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b57e6-50e5-45a0-9436-140bc1b6c78d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:46.728145 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3429c3a-2c36-4dc5-8e63-81d5a419f287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:46.728787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3429c3a-2c36-4dc5-8e63-81d5a419f287 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.729067 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3429c3a-2c36-4dc5-8e63-81d5a419f287 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.729336 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3429c3a-2c36-4dc5-8e63-81d5a419f287 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.730526 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3429c3a-2c36-4dc5-8e63-81d5a419f287 None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:46.731990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3429c3a-2c36-4dc5-8e63-81d5a419f287 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:46.739399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 90/172] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:46 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:46.739399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac4230-0286-4b94-964d-252d9ef9f0bc None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.739399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac4230-0286-4b94-964d-252d9ef9f0bc None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.739399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac4230-0286-4b94-964d-252d9ef9f0bc None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.739399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 91/173] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:46.742564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42ae38-8492-4206-ba9c-7184bf7d16a8 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.742964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42ae38-8492-4206-ba9c-7184bf7d16a8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.743320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de42ae38-8492-4206-ba9c-7184bf7d16a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.756586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884db93-561a-4f89-a09f-8c50fab78139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:46.757211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884db93-561a-4f89-a09f-8c50fab78139 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.757494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884db93-561a-4f89-a09f-8c50fab78139 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.757768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884db93-561a-4f89-a09f-8c50fab78139 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.758482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884db93-561a-4f89-a09f-8c50fab78139 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:46.758791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884db93-561a-4f89-a09f-8c50fab78139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:46.760189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884db93-561a-4f89-a09f-8c50fab78139 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:46.762935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-de42ae38-8492-4206-ba9c-7184bf7d16a8 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:46.764141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-de42ae38-8492-4206-ba9c-7184bf7d16a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:46.769330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 83/174] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:07:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:46.770208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884db93-561a-4f89-a09f-8c50fab78139 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:46.792693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:46.838534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 92/175] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:46.839824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1008c4-a5e5-486b-8694-49300ba90f64 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.840113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1008c4-a5e5-486b-8694-49300ba90f64 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.840391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1008c4-a5e5-486b-8694-49300ba90f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.861120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-bc1008c4-a5e5-486b-8694-49300ba90f64 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:46.862423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-bc1008c4-a5e5-486b-8694-49300ba90f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:46.900607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884db93-561a-4f89-a09f-8c50fab78139 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:46.942296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 93/176] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:46.944064 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c23a03-c4b3-4ba0-a64f-5932f2169fff None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.944490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c23a03-c4b3-4ba0-a64f-5932f2169fff None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.944902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c23a03-c4b3-4ba0-a64f-5932f2169fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.954708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2f761-3f34-483d-b159-e972b12147b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:46.955740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2f761-3f34-483d-b159-e972b12147b3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:46.956221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2f761-3f34-483d-b159-e972b12147b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:46.956684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c2f761-3f34-483d-b159-e972b12147b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:46.957979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2f761-3f34-483d-b159-e972b12147b3 None admin] RBAC: Adding query filter params (role.id=20378c15bf524a4c960b49d2c53cf6b2, scope.project.id=de138855cc8744659425af5f269b8fbc, user.id=1888f4b82c7940fdb9abfe0a57b404c5) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:46.958536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2f761-3f34-483d-b159-e972b12147b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:46.961179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c2f761-3f34-483d-b159-e972b12147b3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:46.972165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 84/177] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:46 2022] GET /identity/v3/role_assignments?user.id=1888f4b82c7940fdb9abfe0a57b404c5&scope.project.id=de138855cc8744659425af5f269b8fbc&role.id=20378c15bf524a4c960b49d2c53cf6b2 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:46.973505 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2f761-3f34-483d-b159-e972b12147b3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:46.976201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e7c23a03-c4b3-4ba0-a64f-5932f2169fff None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:46.979059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e7c23a03-c4b3-4ba0-a64f-5932f2169fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:47.048858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:47.176228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 94/178] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:47.177604 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c23a03-c4b3-4ba0-a64f-5932f2169fff None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:47.209210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:47.224949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c2f761-3f34-483d-b159-e972b12147b3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:47.270646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2d562-3ae9-4e4a-90db-1a298b191ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:47.271330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2d562-3ae9-4e4a-90db-1a298b191ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:47.271586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2d562-3ae9-4e4a-90db-1a298b191ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:47.271844 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2d562-3ae9-4e4a-90db-1a298b191ca6 None admin] PATH_INFO: `/v3/users/b9a31b5d481f471a90714fa648bfed35` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:47.273349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2d562-3ae9-4e4a-90db-1a298b191ca6 None admin] RBAC: Authorizing `identity:get_user(user_id=b9a31b5d481f471a90714fa648bfed35)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:47.276023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2d562-3ae9-4e4a-90db-1a298b191ca6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:47.279412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 85/179] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:46 2022] GET /identity/v3/users/b9a31b5d481f471a90714fa648bfed35 => generated 247 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:47.279412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58a8c-a6cb-4596-9556-221f74bcd8cf None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:47.279757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58a8c-a6cb-4596-9556-221f74bcd8cf None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:47.279896 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea58a8c-a6cb-4596-9556-221f74bcd8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:47.300918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7ea58a8c-a6cb-4596-9556-221f74bcd8cf None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:47.302515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7ea58a8c-a6cb-4596-9556-221f74bcd8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:47.391247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 86/180] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:47.392114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea58a8c-a6cb-4596-9556-221f74bcd8cf None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:47.401254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c23a03-c4b3-4ba0-a64f-5932f2169fff None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:47.484125 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caee0d1-8ac5-41d5-920e-4f31ef812202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:47.484973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caee0d1-8ac5-41d5-920e-4f31ef812202 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:47.485384 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caee0d1-8ac5-41d5-920e-4f31ef812202 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:47.485778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caee0d1-8ac5-41d5-920e-4f31ef812202 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:47.488333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caee0d1-8ac5-41d5-920e-4f31ef812202 None admin] RBAC: Adding query filter params (role.id=aa320f864d914f65bebe5463994ff536, scope.project.id=832b79a184af40d7ad323b5e97816800, user.id=1888f4b82c7940fdb9abfe0a57b404c5) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:47.488950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caee0d1-8ac5-41d5-920e-4f31ef812202 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:47.491469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caee0d1-8ac5-41d5-920e-4f31ef812202 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:47.498532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:47.512162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 95/181] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:47 2022] GET /identity/v3/role_assignments?user.id=1888f4b82c7940fdb9abfe0a57b404c5&scope.project.id=832b79a184af40d7ad323b5e97816800&role.id=aa320f864d914f65bebe5463994ff536 => generated 608 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:47.518621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caee0d1-8ac5-41d5-920e-4f31ef812202 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:47.545319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea58a8c-a6cb-4596-9556-221f74bcd8cf None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:47.597469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:47.606602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7eb29-cea1-481b-8711-03c090fbed6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:47.607261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7eb29-cea1-481b-8711-03c090fbed6a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:47.607540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7eb29-cea1-481b-8711-03c090fbed6a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:47.607812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7eb29-cea1-481b-8711-03c090fbed6a None admin] PATH_INFO: `/v3/users/1888f4b82c7940fdb9abfe0a57b404c5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:47.609636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7eb29-cea1-481b-8711-03c090fbed6a None admin] RBAC: Authorizing `identity:get_user(user_id=1888f4b82c7940fdb9abfe0a57b404c5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:47.611167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7eb29-cea1-481b-8711-03c090fbed6a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:47.613785 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 87/182] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:47 2022] GET /identity/v3/users/1888f4b82c7940fdb9abfe0a57b404c5 => generated 275 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:47.614461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7eb29-cea1-481b-8711-03c090fbed6a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:47.740399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caee0d1-8ac5-41d5-920e-4f31ef812202 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:47.815891 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6451cb-2bfc-4e56-b8b8-818eb690d200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:47.817409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6451cb-2bfc-4e56-b8b8-818eb690d200 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:47.818179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6451cb-2bfc-4e56-b8b8-818eb690d200 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:47.818929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6451cb-2bfc-4e56-b8b8-818eb690d200 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:47.829118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7eb29-cea1-481b-8711-03c090fbed6a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:47.846842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6451cb-2bfc-4e56-b8b8-818eb690d200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4d6451cb-2bfc-4e56-b8b8-818eb690d200 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:47.848132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:47.849706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 96/183] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:47.851296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6451cb-2bfc-4e56-b8b8-818eb690d200 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:47.892036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67d037-b8bb-4dd6-a502-c70dcd1b2a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:47.893124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67d037-b8bb-4dd6-a502-c70dcd1b2a33 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:47.893925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67d037-b8bb-4dd6-a502-c70dcd1b2a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:47.894186 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67d037-b8bb-4dd6-a502-c70dcd1b2a33 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:47.913485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df67d037-b8bb-4dd6-a502-c70dcd1b2a33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-df67d037-b8bb-4dd6-a502-c70dcd1b2a33 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:47.924069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:47.925184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:47.926206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 88/184] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:07:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:47.927597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67d037-b8bb-4dd6-a502-c70dcd1b2a33 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:47.948311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:47.966063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:47.976502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6451cb-2bfc-4e56-b8b8-818eb690d200 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:48.013316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37111b8c-1628-4679-98fc-2f09cf5cdbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:48.013853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37111b8c-1628-4679-98fc-2f09cf5cdbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:48.014165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37111b8c-1628-4679-98fc-2f09cf5cdbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:48.014532 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37111b8c-1628-4679-98fc-2f09cf5cdbd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:48.090900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37111b8c-1628-4679-98fc-2f09cf5cdbd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:48.093261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37111b8c-1628-4679-98fc-2f09cf5cdbd2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:48.096707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67d037-b8bb-4dd6-a502-c70dcd1b2a33 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:48.136467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1347385-ccca-4531-9b25-9051e10681ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:48.137311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1347385-ccca-4531-9b25-9051e10681ac None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:48.137698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1347385-ccca-4531-9b25-9051e10681ac None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:48.138086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1347385-ccca-4531-9b25-9051e10681ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:48.139511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1347385-ccca-4531-9b25-9051e10681ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:48.141217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1347385-ccca-4531-9b25-9051e10681ac None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:48.143080 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 89/185] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:48.144146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1347385-ccca-4531-9b25-9051e10681ac None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:48.212532 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:48.231261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 97/186] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:07:48.232660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37111b8c-1628-4679-98fc-2f09cf5cdbd2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:48.255820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:48.293513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1347385-ccca-4531-9b25-9051e10681ac None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:48.351319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76a573-117f-4a12-b881-f35c99dd08d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:48.352277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76a573-117f-4a12-b881-f35c99dd08d1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:48.352650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76a573-117f-4a12-b881-f35c99dd08d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:48.352972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76a573-117f-4a12-b881-f35c99dd08d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:48.390594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37111b8c-1628-4679-98fc-2f09cf5cdbd2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:48.430797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d191c56-1812-4a8e-bf65-a09ff9819a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:48.431653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d191c56-1812-4a8e-bf65-a09ff9819a9c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:48.431785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d191c56-1812-4a8e-bf65-a09ff9819a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:48.431955 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d191c56-1812-4a8e-bf65-a09ff9819a9c None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:48.432637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d191c56-1812-4a8e-bf65-a09ff9819a9c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:48.432939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d191c56-1812-4a8e-bf65-a09ff9819a9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:48.434375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d191c56-1812-4a8e-bf65-a09ff9819a9c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:48.443622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 98/187] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:07:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:48.444263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d191c56-1812-4a8e-bf65-a09ff9819a9c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:48.451493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76a573-117f-4a12-b881-f35c99dd08d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:48.455013 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76a573-117f-4a12-b881-f35c99dd08d1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:48.541484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:48.551146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:48.577904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 90/188] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:07:48.578972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76a573-117f-4a12-b881-f35c99dd08d1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:48.614364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d191c56-1812-4a8e-bf65-a09ff9819a9c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:48.667140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49388aee-671d-4caa-9877-b2fc517e818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:48.667919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49388aee-671d-4caa-9877-b2fc517e818d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:48.668286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49388aee-671d-4caa-9877-b2fc517e818d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:48.668640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49388aee-671d-4caa-9877-b2fc517e818d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:48.685982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49388aee-671d-4caa-9877-b2fc517e818d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-49388aee-671d-4caa-9877-b2fc517e818d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:48.695741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:48.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 99/189] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:07:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:48.699678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49388aee-671d-4caa-9877-b2fc517e818d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:48.726195 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76a573-117f-4a12-b881-f35c99dd08d1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:48.770864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9dd71-655e-44df-ac0a-80f034c5a38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:48.771472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9dd71-655e-44df-ac0a-80f034c5a38b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:48.771749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9dd71-655e-44df-ac0a-80f034c5a38b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:48.772018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9dd71-655e-44df-ac0a-80f034c5a38b None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:48.772700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9dd71-655e-44df-ac0a-80f034c5a38b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:48.772999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9dd71-655e-44df-ac0a-80f034c5a38b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:48.774429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9dd71-655e-44df-ac0a-80f034c5a38b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:48.783895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 91/190] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:07:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:48.789362 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9dd71-655e-44df-ac0a-80f034c5a38b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:48.835150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:48.842883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49388aee-671d-4caa-9877-b2fc517e818d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:48.868816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:48.904047 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9dd71-655e-44df-ac0a-80f034c5a38b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:48.905724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3b3c6-b033-41d2-970c-99b4663c8e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:48.906809 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3b3c6-b033-41d2-970c-99b4663c8e5d None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:48.907310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3b3c6-b033-41d2-970c-99b4663c8e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:48.907793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca3b3c6-b033-41d2-970c-99b4663c8e5d None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:48.909024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3b3c6-b033-41d2-970c-99b4663c8e5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:48.911840 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca3b3c6-b033-41d2-970c-99b4663c8e5d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:48.941503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 100/191] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:07:48 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:07:48.954353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3b3c6-b033-41d2-970c-99b4663c8e5d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:48.957421 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866f145-076b-4cc8-ad5f-8edf75019a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:48.959763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866f145-076b-4cc8-ad5f-8edf75019a81 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:48.960263 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866f145-076b-4cc8-ad5f-8edf75019a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:48.960737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866f145-076b-4cc8-ad5f-8edf75019a81 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:48.963731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866f145-076b-4cc8-ad5f-8edf75019a81 None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:48.966576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866f145-076b-4cc8-ad5f-8edf75019a81 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:48.970841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 92/192] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:48 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:48.972642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866f145-076b-4cc8-ad5f-8edf75019a81 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:49.120137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca3b3c6-b033-41d2-970c-99b4663c8e5d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:49.157658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866f145-076b-4cc8-ad5f-8edf75019a81 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:49.171734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:49.191534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eec1e6-7aa6-4988-85bc-e6717fa861d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:49.192196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eec1e6-7aa6-4988-85bc-e6717fa861d4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:49.192467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eec1e6-7aa6-4988-85bc-e6717fa861d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:49.192739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eec1e6-7aa6-4988-85bc-e6717fa861d4 None admin] PATH_INFO: `/v3/users/e874b6c6bd824b4b875c5ce31404e4b9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:49.194634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eec1e6-7aa6-4988-85bc-e6717fa861d4 None admin] RBAC: Authorizing `identity:get_user(user_id=e874b6c6bd824b4b875c5ce31404e4b9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:49.197531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eec1e6-7aa6-4988-85bc-e6717fa861d4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:49.200640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 101/193] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:48 2022] GET /identity/v3/users/e874b6c6bd824b4b875c5ce31404e4b9 => generated 289 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:49.202353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eec1e6-7aa6-4988-85bc-e6717fa861d4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:49.215055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef615b-4f93-49f6-9104-1d8c361508cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:49.216118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef615b-4f93-49f6-9104-1d8c361508cd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:49.216574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef615b-4f93-49f6-9104-1d8c361508cd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:49.217033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef615b-4f93-49f6-9104-1d8c361508cd None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:49.218902 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef615b-4f93-49f6-9104-1d8c361508cd None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:49.222084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef615b-4f93-49f6-9104-1d8c361508cd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:49.225940 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 93/194] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:48 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:49.227437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef615b-4f93-49f6-9104-1d8c361508cd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:49.285976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:49.386704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eec1e6-7aa6-4988-85bc-e6717fa861d4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:49.412992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef615b-4f93-49f6-9104-1d8c361508cd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:49.452271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64774a3f-0dbc-4904-a130-b007a352a31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:49.453168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64774a3f-0dbc-4904-a130-b007a352a31a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:49.453616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64774a3f-0dbc-4904-a130-b007a352a31a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:49.454018 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64774a3f-0dbc-4904-a130-b007a352a31a None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:49.454746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64774a3f-0dbc-4904-a130-b007a352a31a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:49.455135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64774a3f-0dbc-4904-a130-b007a352a31a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:49.457394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64774a3f-0dbc-4904-a130-b007a352a31a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:49.468871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 102/195] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:49.469740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64774a3f-0dbc-4904-a130-b007a352a31a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:49.471196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f357caf-2b7e-491a-9fc2-93bb7398b4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:49.472318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f357caf-2b7e-491a-9fc2-93bb7398b4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:49.472833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f357caf-2b7e-491a-9fc2-93bb7398b4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:49.473323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f357caf-2b7e-491a-9fc2-93bb7398b4d5 None admin] PATH_INFO: `/v3/users/450aee69bbe44df982aa0f8ea6872283` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:49.511436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f357caf-2b7e-491a-9fc2-93bb7398b4d5 None admin] RBAC: Authorizing `identity:get_user(user_id=450aee69bbe44df982aa0f8ea6872283)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:49.514199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f357caf-2b7e-491a-9fc2-93bb7398b4d5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:49.517513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 94/196] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:49 2022] GET /identity/v3/users/450aee69bbe44df982aa0f8ea6872283 => generated 283 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:49.519027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f357caf-2b7e-491a-9fc2-93bb7398b4d5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:49.643159 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:49.646161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:49.649614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64774a3f-0dbc-4904-a130-b007a352a31a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:49.721078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7124595-3c0c-4ed3-b86c-d5cbe406cde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:49.722769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7124595-3c0c-4ed3-b86c-d5cbe406cde5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:49.723579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7124595-3c0c-4ed3-b86c-d5cbe406cde5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:49.724276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7124595-3c0c-4ed3-b86c-d5cbe406cde5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:49.726033 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7124595-3c0c-4ed3-b86c-d5cbe406cde5 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=832b79a184af40d7ad323b5e97816800, user.id=b9a31b5d481f471a90714fa648bfed35) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:49.726379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f357caf-2b7e-491a-9fc2-93bb7398b4d5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:49.727135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7124595-3c0c-4ed3-b86c-d5cbe406cde5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:49.730316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7124595-3c0c-4ed3-b86c-d5cbe406cde5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:49.741650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 103/197] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:49 2022] GET /identity/v3/role_assignments?user.id=b9a31b5d481f471a90714fa648bfed35&scope.project.id=832b79a184af40d7ad323b5e97816800&role.id=459d01a218314543a7a8329e88b78331 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:49.743685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7124595-3c0c-4ed3-b86c-d5cbe406cde5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:49.803860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaa6e7-b8cf-4895-8fcf-787942e3aef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:49.804943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaa6e7-b8cf-4895-8fcf-787942e3aef2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:49.805363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaa6e7-b8cf-4895-8fcf-787942e3aef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:49.805852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaa6e7-b8cf-4895-8fcf-787942e3aef2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:49.946906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebaa6e7-b8cf-4895-8fcf-787942e3aef2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:49.948404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebaa6e7-b8cf-4895-8fcf-787942e3aef2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:49.986698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:49.994218 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7124595-3c0c-4ed3-b86c-d5cbe406cde5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:50.054581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffb67e-b734-4b18-b0e6-92d3cd6eb00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:50.055355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffb67e-b734-4b18-b0e6-92d3cd6eb00b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:50.055713 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffb67e-b734-4b18-b0e6-92d3cd6eb00b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:50.056974 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffb67e-b734-4b18-b0e6-92d3cd6eb00b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:50.058353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffb67e-b734-4b18-b0e6-92d3cd6eb00b None admin] RBAC: Adding query filter params (role.id=20378c15bf524a4c960b49d2c53cf6b2, scope.project.id=832b79a184af40d7ad323b5e97816800, user.id=1888f4b82c7940fdb9abfe0a57b404c5) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:50.058764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffb67e-b734-4b18-b0e6-92d3cd6eb00b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:50.061840 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ffb67e-b734-4b18-b0e6-92d3cd6eb00b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:50.075467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 104/198] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:49 2022] GET /identity/v3/role_assignments?user.id=1888f4b82c7940fdb9abfe0a57b404c5&scope.project.id=832b79a184af40d7ad323b5e97816800&role.id=20378c15bf524a4c960b49d2c53cf6b2 => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:50.075467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffb67e-b734-4b18-b0e6-92d3cd6eb00b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:50.107161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 95/199] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:07:50.109051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaa6e7-b8cf-4895-8fcf-787942e3aef2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:50.142173 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:50.294029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffb67e-b734-4b18-b0e6-92d3cd6eb00b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:50.316051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaa6e7-b8cf-4895-8fcf-787942e3aef2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:50.375768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:50.377051 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24530e-8db2-461b-a76d-23a62b85f1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:50.377889 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24530e-8db2-461b-a76d-23a62b85f1be None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:50.378453 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24530e-8db2-461b-a76d-23a62b85f1be None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:50.378911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24530e-8db2-461b-a76d-23a62b85f1be None admin] PATH_INFO: `/v3/users/450aee69bbe44df982aa0f8ea6872283` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:50.383022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d2bec-779e-488e-a218-da7a5606c847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:50.383022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d2bec-779e-488e-a218-da7a5606c847 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:50.383313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d2bec-779e-488e-a218-da7a5606c847 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:50.383607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d2bec-779e-488e-a218-da7a5606c847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:50.384704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24530e-8db2-461b-a76d-23a62b85f1be None admin] RBAC: Authorizing `identity:get_user(user_id=450aee69bbe44df982aa0f8ea6872283)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:50.386408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24530e-8db2-461b-a76d-23a62b85f1be None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:50.394136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 105/200] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:50 2022] GET /identity/v3/users/450aee69bbe44df982aa0f8ea6872283 => generated 283 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:50.395534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24530e-8db2-461b-a76d-23a62b85f1be None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:50.512534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d2bec-779e-488e-a218-da7a5606c847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:50.515994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d2bec-779e-488e-a218-da7a5606c847 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:50.640266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:50.655142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24530e-8db2-461b-a76d-23a62b85f1be None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:50.678006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 96/201] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:07:50.686150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d2bec-779e-488e-a218-da7a5606c847 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:50.729457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee53e0c-5113-4ae5-be53-04104098cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:50.730193 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee53e0c-5113-4ae5-be53-04104098cb70 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:50.730514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee53e0c-5113-4ae5-be53-04104098cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:50.731444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee53e0c-5113-4ae5-be53-04104098cb70 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:50.732827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee53e0c-5113-4ae5-be53-04104098cb70 None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:50.735264 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee53e0c-5113-4ae5-be53-04104098cb70 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:50.738105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 106/202] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:50 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:50.738105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee53e0c-5113-4ae5-be53-04104098cb70 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:50.758849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:50.890204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d2bec-779e-488e-a218-da7a5606c847 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:50.903725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee53e0c-5113-4ae5-be53-04104098cb70 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:50.953011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2f38d-a1e0-4a4e-be13-88358a5d8e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:50.953826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2f38d-a1e0-4a4e-be13-88358a5d8e48 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:50.954497 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2f38d-a1e0-4a4e-be13-88358a5d8e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:50.954888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2f38d-a1e0-4a4e-be13-88358a5d8e48 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:50.956323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2f38d-a1e0-4a4e-be13-88358a5d8e48 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:50.957881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2f38d-a1e0-4a4e-be13-88358a5d8e48 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:50.961088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 107/203] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:50 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:50.962465 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2f38d-a1e0-4a4e-be13-88358a5d8e48 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:50.963909 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0e773-5705-4fd7-a38e-95dcba673f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:50.964650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0e773-5705-4fd7-a38e-95dcba673f8b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:50.965073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0e773-5705-4fd7-a38e-95dcba673f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:50.965463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0e773-5705-4fd7-a38e-95dcba673f8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:51.024746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:51.066777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0e773-5705-4fd7-a38e-95dcba673f8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:51.069560 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0e773-5705-4fd7-a38e-95dcba673f8b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:51.112478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:51.168831 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2f38d-a1e0-4a4e-be13-88358a5d8e48 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:51.204085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 97/204] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:07:51.209423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0e773-5705-4fd7-a38e-95dcba673f8b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:51.241309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57244aab-0ecb-4b18-a9aa-f3eb765459fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:51.241925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57244aab-0ecb-4b18-a9aa-f3eb765459fa None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:51.242260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57244aab-0ecb-4b18-a9aa-f3eb765459fa None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:51.242488 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57244aab-0ecb-4b18-a9aa-f3eb765459fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:51.243231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57244aab-0ecb-4b18-a9aa-f3eb765459fa None admin] RBAC: Adding query filter params (role.id=bbbfb342bd244610906d836359f61fcf, scope.project.id=832b79a184af40d7ad323b5e97816800, user.id=e874b6c6bd824b4b875c5ce31404e4b9) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:51.243535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57244aab-0ecb-4b18-a9aa-f3eb765459fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:51.244963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57244aab-0ecb-4b18-a9aa-f3eb765459fa None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:51.266431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 108/205] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:50 2022] GET /identity/v3/role_assignments?user.id=e874b6c6bd824b4b875c5ce31404e4b9&scope.project.id=832b79a184af40d7ad323b5e97816800&role.id=bbbfb342bd244610906d836359f61fcf => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:51.267473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57244aab-0ecb-4b18-a9aa-f3eb765459fa None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:51.353241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0e773-5705-4fd7-a38e-95dcba673f8b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:51.393898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:51.403237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393777f-29f6-490b-a775-b27479532e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:51.404297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393777f-29f6-490b-a775-b27479532e53 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:51.404849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393777f-29f6-490b-a775-b27479532e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:51.405943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4393777f-29f6-490b-a775-b27479532e53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:51.407052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393777f-29f6-490b-a775-b27479532e53 None admin] RBAC: Adding query filter params (role.id=aa320f864d914f65bebe5463994ff536, scope.project.id=6753f2523be140a3a48348df431cf0d9, user.id=450aee69bbe44df982aa0f8ea6872283) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:51.408523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393777f-29f6-490b-a775-b27479532e53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:51.410695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4393777f-29f6-490b-a775-b27479532e53 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:51.419981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 98/206] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:51 2022] GET /identity/v3/role_assignments?user.id=450aee69bbe44df982aa0f8ea6872283&scope.project.id=6753f2523be140a3a48348df431cf0d9&role.id=aa320f864d914f65bebe5463994ff536 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:51.469514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:51.477644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57244aab-0ecb-4b18-a9aa-f3eb765459fa None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:51.539225 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e95e6b-4c16-4b76-b81b-b1041e618d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:51.539934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e95e6b-4c16-4b76-b81b-b1041e618d70 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:51.540192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e95e6b-4c16-4b76-b81b-b1041e618d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:51.540484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e95e6b-4c16-4b76-b81b-b1041e618d70 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:51.543486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e95e6b-4c16-4b76-b81b-b1041e618d70 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:51.545711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e95e6b-4c16-4b76-b81b-b1041e618d70 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:51.548143 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 109/207] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:51 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:51.557409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393777f-29f6-490b-a775-b27479532e53 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:51.692090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4393777f-29f6-490b-a775-b27479532e53 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:51.739720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a88b04-eeaf-4f3a-8314-ed8aaeead8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:51.740704 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a88b04-eeaf-4f3a-8314-ed8aaeead8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:51.741196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a88b04-eeaf-4f3a-8314-ed8aaeead8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:51.741702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a88b04-eeaf-4f3a-8314-ed8aaeead8f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:51.743511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a88b04-eeaf-4f3a-8314-ed8aaeead8f4 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=6753f2523be140a3a48348df431cf0d9, user.id=450aee69bbe44df982aa0f8ea6872283) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:51.744066 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a88b04-eeaf-4f3a-8314-ed8aaeead8f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:51.745767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a88b04-eeaf-4f3a-8314-ed8aaeead8f4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:51.758365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 99/208] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:51 2022] GET /identity/v3/role_assignments?user.id=450aee69bbe44df982aa0f8ea6872283&scope.project.id=6753f2523be140a3a48348df431cf0d9&role.id=459d01a218314543a7a8329e88b78331 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:52.191117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254ac43-71fb-46e7-8161-7fac2cbef314 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:52.191668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254ac43-71fb-46e7-8161-7fac2cbef314 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:52.191668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8254ac43-71fb-46e7-8161-7fac2cbef314 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:52.193004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 110/209] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:52.198811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d46430-bf4d-45bf-8974-89deb658db36 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:52.199305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d46430-bf4d-45bf-8974-89deb658db36 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:52.199394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d46430-bf4d-45bf-8974-89deb658db36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:52.219244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-09d46430-bf4d-45bf-8974-89deb658db36 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:52.220456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-09d46430-bf4d-45bf-8974-89deb658db36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:52.287533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:52.307548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 100/210] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:52.313410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbf7cc-57ff-4793-ab5d-e426e14dc6c7 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:52.313897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbf7cc-57ff-4793-ab5d-e426e14dc6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:52.314463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cbf7cc-57ff-4793-ab5d-e426e14dc6c7 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:52.316995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 111/211] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:52.326243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47395f96-23a8-484d-a80c-85f90df9ce29 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:52.327528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47395f96-23a8-484d-a80c-85f90df9ce29 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:52.327998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47395f96-23a8-484d-a80c-85f90df9ce29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:52.362594 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-47395f96-23a8-484d-a80c-85f90df9ce29 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:52.364636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-47395f96-23a8-484d-a80c-85f90df9ce29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:52.470362 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 101/212] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:07:52.753438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbf7cc-57ff-4793-ab5d-e426e14dc6c7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:52.806786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47395f96-23a8-484d-a80c-85f90df9ce29 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:52.929620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cbf7cc-57ff-4793-ab5d-e426e14dc6c7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:52.991718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47395f96-23a8-484d-a80c-85f90df9ce29 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:52.994429 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c8823-48e6-4e93-9e85-08eca33ca398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:52.995666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c8823-48e6-4e93-9e85-08eca33ca398 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:52.996174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c8823-48e6-4e93-9e85-08eca33ca398 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:52.996698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c8823-48e6-4e93-9e85-08eca33ca398 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:53.017368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c8823-48e6-4e93-9e85-08eca33ca398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-519c8823-48e6-4e93-9e85-08eca33ca398 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:53.027918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:53.028850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:53.029694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:53.029694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:53.029694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:53.029694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:53.029694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:53.029694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:53.029694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:53.029694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:53.029694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:53.029694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:53.029694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:53.029694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:07:53.030884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:53.034131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 112/213] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:53.037588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c8823-48e6-4e93-9e85-08eca33ca398 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:53.056837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8163203-9b61-4f37-9fa7-396f0cb8f88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:53.057938 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8163203-9b61-4f37-9fa7-396f0cb8f88f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:53.058482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8163203-9b61-4f37-9fa7-396f0cb8f88f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:53.058960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8163203-9b61-4f37-9fa7-396f0cb8f88f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:53.061517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8163203-9b61-4f37-9fa7-396f0cb8f88f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:53.064513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8163203-9b61-4f37-9fa7-396f0cb8f88f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:53.068032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 102/214] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:53.068739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:53.080534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8163203-9b61-4f37-9fa7-396f0cb8f88f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:53.174440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c8823-48e6-4e93-9e85-08eca33ca398 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:53.236747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0a5b2-8347-4b5f-ad02-89837a218981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:53.237772 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0a5b2-8347-4b5f-ad02-89837a218981 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:53.237916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0a5b2-8347-4b5f-ad02-89837a218981 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:53.238337 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0a5b2-8347-4b5f-ad02-89837a218981 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:53.239171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0a5b2-8347-4b5f-ad02-89837a218981 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:53.239599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0a5b2-8347-4b5f-ad02-89837a218981 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:53.241219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0a5b2-8347-4b5f-ad02-89837a218981 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:53.256467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 113/215] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:07:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:53.257636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0a5b2-8347-4b5f-ad02-89837a218981 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:53.258172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8163203-9b61-4f37-9fa7-396f0cb8f88f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:53.315563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ce250-dc11-476b-94b2-a08d2e451bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:53.316283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ce250-dc11-476b-94b2-a08d2e451bda None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:53.316989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ce250-dc11-476b-94b2-a08d2e451bda None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:53.317376 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ce250-dc11-476b-94b2-a08d2e451bda None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:53.318792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ce250-dc11-476b-94b2-a08d2e451bda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:53.320763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ce250-dc11-476b-94b2-a08d2e451bda None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:53.355637 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 103/216] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:07:53 2022] POST /identity/v3/users => generated 297 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:07:53.440942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:53.473308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0a5b2-8347-4b5f-ad02-89837a218981 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:53.519663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2408083-e89d-41b9-a785-89679b61d88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:53.520344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2408083-e89d-41b9-a785-89679b61d88f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:53.520620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2408083-e89d-41b9-a785-89679b61d88f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:53.520895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2408083-e89d-41b9-a785-89679b61d88f None admin] PATH_INFO: `/v3/users/1e3486cfff3c4193b5a188a9bc5a211d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:53.537636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2408083-e89d-41b9-a785-89679b61d88f None admin] RBAC: Authorizing `identity:get_user(user_id=1e3486cfff3c4193b5a188a9bc5a211d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:53.539230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2408083-e89d-41b9-a785-89679b61d88f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:53.541275 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 114/217] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:53 2022] GET /identity/v3/users/1e3486cfff3c4193b5a188a9bc5a211d => generated 297 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:53.547199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ce250-dc11-476b-94b2-a08d2e451bda None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:53.604473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:53.672393 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ce250-dc11-476b-94b2-a08d2e451bda None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:53.718139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd9b57-b2a0-428b-9e8b-21fc88c07fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:53.718824 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd9b57-b2a0-428b-9e8b-21fc88c07fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:53.719109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd9b57-b2a0-428b-9e8b-21fc88c07fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:53.719459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd9b57-b2a0-428b-9e8b-21fc88c07fb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:53.806531 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd9b57-b2a0-428b-9e8b-21fc88c07fb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:53.808212 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cd9b57-b2a0-428b-9e8b-21fc88c07fb8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:53.838998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332910d-a806-450a-acb3-6823e3080db4 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:53.839345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332910d-a806-450a-acb3-6823e3080db4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:53.839551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332910d-a806-450a-acb3-6823e3080db4 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:53.841420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 115/218] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:07:53.850903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170c717-1e6c-4fe0-ba0f-770adb993632 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:53.851221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170c717-1e6c-4fe0-ba0f-770adb993632 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:53.851467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170c717-1e6c-4fe0-ba0f-770adb993632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:53.874031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:53.894625 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2170c717-1e6c-4fe0-ba0f-770adb993632 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:53.896706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2170c717-1e6c-4fe0-ba0f-770adb993632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:53.956887 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 104/219] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:07:53.965427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd9b57-b2a0-428b-9e8b-21fc88c07fb8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:54.005945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 116/220] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:54.098337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd9b57-b2a0-428b-9e8b-21fc88c07fb8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:54.138860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e307d82-2ca9-4ab3-9d13-7ca5d15fe98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:54.139775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e307d82-2ca9-4ab3-9d13-7ca5d15fe98d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:54.140206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e307d82-2ca9-4ab3-9d13-7ca5d15fe98d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:54.140645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e307d82-2ca9-4ab3-9d13-7ca5d15fe98d None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:54.142027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e307d82-2ca9-4ab3-9d13-7ca5d15fe98d None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:54.143706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e307d82-2ca9-4ab3-9d13-7ca5d15fe98d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:54.145602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 105/221] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:53 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:54.151935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170c717-1e6c-4fe0-ba0f-770adb993632 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:54.241373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:54.272463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170c717-1e6c-4fe0-ba0f-770adb993632 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:54.310750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8ee46-1c47-41ae-8d8c-9ba03e86a44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:54.312167 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8ee46-1c47-41ae-8d8c-9ba03e86a44e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:54.312596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8ee46-1c47-41ae-8d8c-9ba03e86a44e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:54.313050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8ee46-1c47-41ae-8d8c-9ba03e86a44e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:54.313961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8ee46-1c47-41ae-8d8c-9ba03e86a44e None admin] RBAC: Adding query filter params (role.id=20378c15bf524a4c960b49d2c53cf6b2, scope.project.id=6753f2523be140a3a48348df431cf0d9, user.id=1e3486cfff3c4193b5a188a9bc5a211d) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:54.314448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8ee46-1c47-41ae-8d8c-9ba03e86a44e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:54.316090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8ee46-1c47-41ae-8d8c-9ba03e86a44e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:54.323821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 117/222] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:54 2022] GET /identity/v3/role_assignments?user.id=1e3486cfff3c4193b5a188a9bc5a211d&scope.project.id=6753f2523be140a3a48348df431cf0d9&role.id=20378c15bf524a4c960b49d2c53cf6b2 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:54.454152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e307d82-2ca9-4ab3-9d13-7ca5d15fe98d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:54.472806 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:54.593154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e307d82-2ca9-4ab3-9d13-7ca5d15fe98d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:54.632824 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abee1b-1cbe-4870-81d3-ac2088ade083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:54.633766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abee1b-1cbe-4870-81d3-ac2088ade083 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:54.634231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abee1b-1cbe-4870-81d3-ac2088ade083 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:54.634651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53abee1b-1cbe-4870-81d3-ac2088ade083 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:54.647596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53abee1b-1cbe-4870-81d3-ac2088ade083 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-53abee1b-1cbe-4870-81d3-ac2088ade083 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:54.652800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:54.653755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:54.655485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:54.655485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:54.655485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:54.655485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:54.655485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:54.655485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:54.655485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:54.655485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:54.655485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:54.655485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:54.655485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:07:54.655485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:54.657816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 106/223] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:07:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:54.664100 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8ee46-1c47-41ae-8d8c-9ba03e86a44e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:54.787097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8ee46-1c47-41ae-8d8c-9ba03e86a44e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:54.825529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a643a-b809-4b7a-bafa-a6758e7bb41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:54.826499 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a643a-b809-4b7a-bafa-a6758e7bb41d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:54.826928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a643a-b809-4b7a-bafa-a6758e7bb41d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:54.827610 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a643a-b809-4b7a-bafa-a6758e7bb41d None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:54.828510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a643a-b809-4b7a-bafa-a6758e7bb41d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:54.828962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a643a-b809-4b7a-bafa-a6758e7bb41d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:54.830570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a643a-b809-4b7a-bafa-a6758e7bb41d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:54.836956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:54.841101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 118/224] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:07:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:54.847661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abee1b-1cbe-4870-81d3-ac2088ade083 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:54.891078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e62bd2-08e6-4fb2-ae09-c9ec31b9107e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:54.891541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e62bd2-08e6-4fb2-ae09-c9ec31b9107e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:54.892003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e62bd2-08e6-4fb2-ae09-c9ec31b9107e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:54.894316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 119/225] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:07:54.902997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebd381-f0f7-4500-8787-7d4173f06548 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:54.903459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebd381-f0f7-4500-8787-7d4173f06548 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:54.903901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ebd381-f0f7-4500-8787-7d4173f06548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:54.947226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d4ebd381-f0f7-4500-8787-7d4173f06548 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:54.958263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d4ebd381-f0f7-4500-8787-7d4173f06548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:54.989171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:55.029028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53abee1b-1cbe-4870-81d3-ac2088ade083 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:55.095150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b714649-f4aa-4a69-9daf-bc5125c4760e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:55.096271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b714649-f4aa-4a69-9daf-bc5125c4760e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:55.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b714649-f4aa-4a69-9daf-bc5125c4760e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:55.097167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b714649-f4aa-4a69-9daf-bc5125c4760e None admin] PATH_INFO: `/v3/users/450aee69bbe44df982aa0f8ea6872283` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:55.099042 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b714649-f4aa-4a69-9daf-bc5125c4760e None admin] RBAC: Authorizing `identity:get_user(user_id=450aee69bbe44df982aa0f8ea6872283)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:55.101706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b714649-f4aa-4a69-9daf-bc5125c4760e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:55.105032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 107/226] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:54 2022] GET /identity/v3/users/450aee69bbe44df982aa0f8ea6872283 => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:55.111539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b714649-f4aa-4a69-9daf-bc5125c4760e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:55.221428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 120/227] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:55.309686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b714649-f4aa-4a69-9daf-bc5125c4760e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:55.383256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fb43b-3509-4ea7-ae71-dc4835bdab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:55.384344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fb43b-3509-4ea7-ae71-dc4835bdab03 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:55.384825 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fb43b-3509-4ea7-ae71-dc4835bdab03 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:55.385319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295fb43b-3509-4ea7-ae71-dc4835bdab03 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:55.387342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fb43b-3509-4ea7-ae71-dc4835bdab03 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:55.390207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295fb43b-3509-4ea7-ae71-dc4835bdab03 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:55.393432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 108/228] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:55 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:07:55.400437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebd381-f0f7-4500-8787-7d4173f06548 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:55.595613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ebd381-f0f7-4500-8787-7d4173f06548 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:55.630401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:55.637496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c988fe-5dc1-4148-9ff3-2e06ac891a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:55.638140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c988fe-5dc1-4148-9ff3-2e06ac891a85 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:55.638481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c988fe-5dc1-4148-9ff3-2e06ac891a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:55.638918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c988fe-5dc1-4148-9ff3-2e06ac891a85 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9/users/450aee69bbe44df982aa0f8ea6872283/roles/aa320f864d914f65bebe5463994ff536` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:55.640235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fb43b-3509-4ea7-ae71-dc4835bdab03 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:55.642400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c988fe-5dc1-4148-9ff3-2e06ac891a85 None admin] RBAC: Authorizing `identity:create_grant(project_id=6753f2523be140a3a48348df431cf0d9, user_id=450aee69bbe44df982aa0f8ea6872283, role_id=aa320f864d914f65bebe5463994ff536)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:55.646394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c988fe-5dc1-4148-9ff3-2e06ac891a85 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:55.669130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:55.670742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 121/229] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:07:55 2022] PUT /identity/v3/projects/6753f2523be140a3a48348df431cf0d9/users/450aee69bbe44df982aa0f8ea6872283/roles/aa320f864d914f65bebe5463994ff536 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:07:55.858721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295fb43b-3509-4ea7-ae71-dc4835bdab03 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:55.927522 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfedecc-ead8-48d2-97b8-a0d7660478ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:55.929065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfedecc-ead8-48d2-97b8-a0d7660478ee None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:55.929796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfedecc-ead8-48d2-97b8-a0d7660478ee None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:55.930540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfedecc-ead8-48d2-97b8-a0d7660478ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:55.951652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfedecc-ead8-48d2-97b8-a0d7660478ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-adfedecc-ead8-48d2-97b8-a0d7660478ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:55.962923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:55.964628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:55.968707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:55.968707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:55.968707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:55.968707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:55.968707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:55.968707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:55.968707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:55.968707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:55.968707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:55.968707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:55.968707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:07:55.968707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:55.973081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 109/230] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:07:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:55.980527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c988fe-5dc1-4148-9ff3-2e06ac891a85 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:56.142351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c988fe-5dc1-4148-9ff3-2e06ac891a85 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:56.186820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfd9d9-70d5-40b4-ace3-e65941bc8008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:56.187399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfd9d9-70d5-40b4-ace3-e65941bc8008 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:56.187696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfd9d9-70d5-40b4-ace3-e65941bc8008 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:56.188031 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cfd9d9-70d5-40b4-ace3-e65941bc8008 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:56.188786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfd9d9-70d5-40b4-ace3-e65941bc8008 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:56.189097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfd9d9-70d5-40b4-ace3-e65941bc8008 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:56.190533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cfd9d9-70d5-40b4-ace3-e65941bc8008 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:56.200628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 122/231] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:56.209489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfedecc-ead8-48d2-97b8-a0d7660478ee None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:56.328816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:56.406343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfedecc-ead8-48d2-97b8-a0d7660478ee None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:56.469814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26a3fb-1e9e-4368-99a5-e32a4f2eafca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:56.470389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26a3fb-1e9e-4368-99a5-e32a4f2eafca None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:56.470671 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26a3fb-1e9e-4368-99a5-e32a4f2eafca None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:56.470948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26a3fb-1e9e-4368-99a5-e32a4f2eafca None admin] PATH_INFO: `/v3/users/450aee69bbe44df982aa0f8ea6872283` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:56.472134 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26a3fb-1e9e-4368-99a5-e32a4f2eafca None admin] RBAC: Authorizing `identity:get_user(user_id=450aee69bbe44df982aa0f8ea6872283)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:56.473598 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26a3fb-1e9e-4368-99a5-e32a4f2eafca None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:56.475248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 110/232] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:56 2022] GET /identity/v3/users/450aee69bbe44df982aa0f8ea6872283 => generated 283 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:56.481188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfd9d9-70d5-40b4-ace3-e65941bc8008 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:56.611199 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:56.724375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cfd9d9-70d5-40b4-ace3-e65941bc8008 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:56.767418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598c363-329d-41b2-9ee4-8e6d72bb8874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:56.768227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598c363-329d-41b2-9ee4-8e6d72bb8874 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:56.768582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598c363-329d-41b2-9ee4-8e6d72bb8874 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:56.768857 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598c363-329d-41b2-9ee4-8e6d72bb8874 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:56.770099 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598c363-329d-41b2-9ee4-8e6d72bb8874 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:56.771565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598c363-329d-41b2-9ee4-8e6d72bb8874 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:56.773561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 123/233] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:56 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:56.779495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26a3fb-1e9e-4368-99a5-e32a4f2eafca None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:56.915246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26a3fb-1e9e-4368-99a5-e32a4f2eafca None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:56.971781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e4b81-45a3-4aa0-83ee-45465db81b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:56.972485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e4b81-45a3-4aa0-83ee-45465db81b1a None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:56.972814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e4b81-45a3-4aa0-83ee-45465db81b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:56.973101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e4b81-45a3-4aa0-83ee-45465db81b1a None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9/users/450aee69bbe44df982aa0f8ea6872283/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:56.975283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e4b81-45a3-4aa0-83ee-45465db81b1a None admin] RBAC: Authorizing `identity:create_grant(project_id=6753f2523be140a3a48348df431cf0d9, user_id=450aee69bbe44df982aa0f8ea6872283, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:56.976772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e4b81-45a3-4aa0-83ee-45465db81b1a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:56.983588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:57.003337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 111/234] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:07:56 2022] PUT /identity/v3/projects/6753f2523be140a3a48348df431cf0d9/users/450aee69bbe44df982aa0f8ea6872283/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:07:57.216687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31906a25-87b8-4ec8-9803-d6cb2e9f297f None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:57.218073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31906a25-87b8-4ec8-9803-d6cb2e9f297f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:57.218597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31906a25-87b8-4ec8-9803-d6cb2e9f297f None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:57.220789 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 124/235] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:07:57.229040 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b186d16-7004-4149-9bd7-761b376a4598 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:57.229676 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b186d16-7004-4149-9bd7-761b376a4598 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:57.230214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b186d16-7004-4149-9bd7-761b376a4598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:57.257331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7b186d16-7004-4149-9bd7-761b376a4598 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:57.259236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7b186d16-7004-4149-9bd7-761b376a4598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:57.272722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f3829-f4d6-4800-a23a-98202f09f327 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:57.273593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f3829-f4d6-4800-a23a-98202f09f327 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:57.274321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f3829-f4d6-4800-a23a-98202f09f327 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:57.277397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 125/236] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:57.287094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613d5b4-0410-4b86-b9e6-75daa07f82bf None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:57.287548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613d5b4-0410-4b86-b9e6-75daa07f82bf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:57.288021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3613d5b4-0410-4b86-b9e6-75daa07f82bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:57.323321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3613d5b4-0410-4b86-b9e6-75daa07f82bf None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:57.326283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3613d5b4-0410-4b86-b9e6-75daa07f82bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:57.355915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 112/237] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:57.461852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 126/238] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:57.844669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b186d16-7004-4149-9bd7-761b376a4598 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:58.033159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b186d16-7004-4149-9bd7-761b376a4598 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:58.054565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613d5b4-0410-4b86-b9e6-75daa07f82bf None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:58.097029 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:58.129516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e330f10-e9a3-421a-bff8-8768359e8068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:58.130668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e330f10-e9a3-421a-bff8-8768359e8068 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:58.131161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e330f10-e9a3-421a-bff8-8768359e8068 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:58.131648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e330f10-e9a3-421a-bff8-8768359e8068 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:58.171482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e330f10-e9a3-421a-bff8-8768359e8068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-2e330f10-e9a3-421a-bff8-8768359e8068 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:58.187650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:58.188600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:58.189397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:58.189397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:58.189397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:58.189397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:58.189397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:58.189397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:58.189397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:58.189397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:58.189397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:58.189397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:58.189397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:07:58.189397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:07:58.190414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 113/239] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:58.190479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:58.197171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e330f10-e9a3-421a-bff8-8768359e8068 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:58.297158 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3613d5b4-0410-4b86-b9e6-75daa07f82bf None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:58.362007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e47d7c2-3da4-496b-bbdb-a330c33319b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:58.362931 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47d7c2-3da4-496b-bbdb-a330c33319b1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:58.364021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47d7c2-3da4-496b-bbdb-a330c33319b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:58.364484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e47d7c2-3da4-496b-bbdb-a330c33319b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:58.366198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47d7c2-3da4-496b-bbdb-a330c33319b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:58.367819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e47d7c2-3da4-496b-bbdb-a330c33319b1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:58.370612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 127/240] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:07:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:58.372554 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5b590-8184-4ea2-96c8-af276e50fbeb None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:58.373045 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5b590-8184-4ea2-96c8-af276e50fbeb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:58.374039 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5b590-8184-4ea2-96c8-af276e50fbeb None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:58.380661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 128/241] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:07:58.380661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5b590-8184-4ea2-96c8-af276e50fbeb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:58.468873 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e330f10-e9a3-421a-bff8-8768359e8068 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:58.531699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5b590-8184-4ea2-96c8-af276e50fbeb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:58.541322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679629ee-f195-4e7d-a402-effd8df74b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:58.542864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679629ee-f195-4e7d-a402-effd8df74b47 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:58.543609 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679629ee-f195-4e7d-a402-effd8df74b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:58.544423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679629ee-f195-4e7d-a402-effd8df74b47 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:58.546459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679629ee-f195-4e7d-a402-effd8df74b47 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:07:58.547014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679629ee-f195-4e7d-a402-effd8df74b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:58.554127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679629ee-f195-4e7d-a402-effd8df74b47 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:58.568637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:58.569479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 114/242] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:07:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:58.571880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5712a7-54fc-42ef-bb9a-35205eb069d5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:58.572649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5712a7-54fc-42ef-bb9a-35205eb069d5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:58.573278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5712a7-54fc-42ef-bb9a-35205eb069d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:58.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc7dce-2570-488e-8592-d0a5997d1ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:58.603515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc7dce-2570-488e-8592-d0a5997d1ffe None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:58.604126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc7dce-2570-488e-8592-d0a5997d1ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:58.604576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc7dce-2570-488e-8592-d0a5997d1ffe None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:58.605631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc7dce-2570-488e-8592-d0a5997d1ffe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:58.608457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc7dce-2570-488e-8592-d0a5997d1ffe None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:58.610537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0c5712a7-54fc-42ef-bb9a-35205eb069d5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:58.612827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0c5712a7-54fc-42ef-bb9a-35205eb069d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:58.648827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 129/243] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:07:58 2022] POST /identity/v3/users => generated 293 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:07:58.650340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc7dce-2570-488e-8592-d0a5997d1ffe None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:58.708124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:58.778654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 115/244] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:58.911947 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc7dce-2570-488e-8592-d0a5997d1ffe None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:58.984292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720523d5-4697-4ca7-ab36-4e16932de574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:58.985365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720523d5-4697-4ca7-ab36-4e16932de574 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:58.985911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720523d5-4697-4ca7-ab36-4e16932de574 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:58.986423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720523d5-4697-4ca7-ab36-4e16932de574 None admin] PATH_INFO: `/v3/users/f29809ecb6ed495c9e4b687f2afaa41f` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:59.015835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720523d5-4697-4ca7-ab36-4e16932de574 None admin] RBAC: Authorizing `identity:get_user(user_id=f29809ecb6ed495c9e4b687f2afaa41f)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:59.018671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720523d5-4697-4ca7-ab36-4e16932de574 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:59.021903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 130/245] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:07:58 2022] GET /identity/v3/users/f29809ecb6ed495c9e4b687f2afaa41f => generated 297 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:59.028589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5712a7-54fc-42ef-bb9a-35205eb069d5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:59.225954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5712a7-54fc-42ef-bb9a-35205eb069d5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:59.272392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c218f-cf7d-4377-baee-0e721c6505d0 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:59.273204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c218f-cf7d-4377-baee-0e721c6505d0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:59.274200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c218f-cf7d-4377-baee-0e721c6505d0 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:59.278038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 131/246] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:07:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:07:59.284494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d917ea-ae2c-4aeb-8153-fc7f5a293119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:59.285130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d917ea-ae2c-4aeb-8153-fc7f5a293119 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:59.285413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d917ea-ae2c-4aeb-8153-fc7f5a293119 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:59.285684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d917ea-ae2c-4aeb-8153-fc7f5a293119 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:59.292523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca229d-d1d0-4da9-a895-a86b58a11230 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:59.294292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca229d-d1d0-4da9-a895-a86b58a11230 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:59.294481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ca229d-d1d0-4da9-a895-a86b58a11230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:59.332740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-47ca229d-d1d0-4da9-a895-a86b58a11230 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:07:59.339125 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-47ca229d-d1d0-4da9-a895-a86b58a11230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:07:59.350743 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:59.385400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d917ea-ae2c-4aeb-8153-fc7f5a293119 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:59.386924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d917ea-ae2c-4aeb-8153-fc7f5a293119 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:59.410008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:59.517727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 132/247] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:07:59.522698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 116/248] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:07:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 16:07:59.522985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca229d-d1d0-4da9-a895-a86b58a11230 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:59.532853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d917ea-ae2c-4aeb-8153-fc7f5a293119 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:59.688841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:59.727316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ca229d-d1d0-4da9-a895-a86b58a11230 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:59.777926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d917ea-ae2c-4aeb-8153-fc7f5a293119 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:07:59.794476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1d0a0-78e5-4b7b-aac5-6f90dc0dd5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:59.796557 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1d0a0-78e5-4b7b-aac5-6f90dc0dd5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:59.797406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1d0a0-78e5-4b7b-aac5-6f90dc0dd5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:59.798157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1d0a0-78e5-4b7b-aac5-6f90dc0dd5f8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:59.821436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f1d0a0-78e5-4b7b-aac5-6f90dc0dd5f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-12f1d0a0-78e5-4b7b-aac5-6f90dc0dd5f8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:07:59.832660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:07:59.834416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:07:59.835943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:07:59.835943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:07:59.835943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:07:59.835943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:07:59.835943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:07:59.835943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:07:59.835943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:07:59.835943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:07:59.835943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:07:59.835943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:07:59.835943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:07:59.835943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:07:59.841783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 133/249] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:07:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:07:59.851402 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1d0a0-78e5-4b7b-aac5-6f90dc0dd5f8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:07:59.900193 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:07:59.927314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b371a-5b58-450f-8c95-69cf0c887814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:07:59.928397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b371a-5b58-450f-8c95-69cf0c887814 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:07:59.928882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b371a-5b58-450f-8c95-69cf0c887814 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:07:59.929361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b371a-5b58-450f-8c95-69cf0c887814 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:07:59.931497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b371a-5b58-450f-8c95-69cf0c887814 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:07:59.934636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b371a-5b58-450f-8c95-69cf0c887814 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:07:59.938064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 117/250] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:07:59 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:07:59.952543 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b371a-5b58-450f-8c95-69cf0c887814 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:00.057486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f1d0a0-78e5-4b7b-aac5-6f90dc0dd5f8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:00.125693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c22a6-a173-4d86-a4a6-9999568d5f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:00.127301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c22a6-a173-4d86-a4a6-9999568d5f7d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:00.128144 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c22a6-a173-4d86-a4a6-9999568d5f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:00.128984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c22a6-a173-4d86-a4a6-9999568d5f7d None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:00.130448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c22a6-a173-4d86-a4a6-9999568d5f7d None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:00.131264 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c22a6-a173-4d86-a4a6-9999568d5f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:00.134593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c22a6-a173-4d86-a4a6-9999568d5f7d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:00.151216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 134/251] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:07:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:00.158089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c22a6-a173-4d86-a4a6-9999568d5f7d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:00.326644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:00.339988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b371a-5b58-450f-8c95-69cf0c887814 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:00.357981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c22a6-a173-4d86-a4a6-9999568d5f7d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:00.423334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307146c-4c72-42ab-a16d-0eaa4f5ddad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:00.424926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307146c-4c72-42ab-a16d-0eaa4f5ddad7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:00.425782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307146c-4c72-42ab-a16d-0eaa4f5ddad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:00.426649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307146c-4c72-42ab-a16d-0eaa4f5ddad7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:00.441786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd000d0e-aa43-4ee1-b9a6-e5735706c962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:00.445738 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd000d0e-aa43-4ee1-b9a6-e5735706c962 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:00.448859 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd000d0e-aa43-4ee1-b9a6-e5735706c962 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:00.455037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd000d0e-aa43-4ee1-b9a6-e5735706c962 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:00.456877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd000d0e-aa43-4ee1-b9a6-e5735706c962 None admin] RBAC: Adding query filter params (role.id=bbbfb342bd244610906d836359f61fcf, scope.project.id=6753f2523be140a3a48348df431cf0d9, user.id=f29809ecb6ed495c9e4b687f2afaa41f) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:00.457625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd000d0e-aa43-4ee1-b9a6-e5735706c962 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:00.460417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd000d0e-aa43-4ee1-b9a6-e5735706c962 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:00.462319 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307146c-4c72-42ab-a16d-0eaa4f5ddad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-c307146c-4c72-42ab-a16d-0eaa4f5ddad7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:00.476886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:00.478880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:00.480602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:00.480602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:00.480602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:00.480602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:00.480602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:00.480602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:00.480602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:00.480602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:00.480602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:00.480602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:00.480602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:00.480602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:00.489862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 118/252] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:07:59 2022] GET /identity/v3/role_assignments?user.id=f29809ecb6ed495c9e4b687f2afaa41f&scope.project.id=6753f2523be140a3a48348df431cf0d9&role.id=bbbfb342bd244610906d836359f61fcf => generated 259 bytes in 539 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:00.491645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd000d0e-aa43-4ee1-b9a6-e5735706c962 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:00.494210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 135/253] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:08:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:00.501292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307146c-4c72-42ab-a16d-0eaa4f5ddad7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:00.543897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:00.753714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307146c-4c72-42ab-a16d-0eaa4f5ddad7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:00.782570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd000d0e-aa43-4ee1-b9a6-e5735706c962 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:00.826484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40b09a-9450-47d0-aab8-97ddb292b93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:00.827816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40b09a-9450-47d0-aab8-97ddb292b93e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:00.828289 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40b09a-9450-47d0-aab8-97ddb292b93e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:00.828763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40b09a-9450-47d0-aab8-97ddb292b93e None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:00.830005 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40b09a-9450-47d0-aab8-97ddb292b93e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:00.830597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40b09a-9450-47d0-aab8-97ddb292b93e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:00.833150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40b09a-9450-47d0-aab8-97ddb292b93e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:00.863222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf63c6b9-6b11-4466-bd5b-cb676d191393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:00.864250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63c6b9-6b11-4466-bd5b-cb676d191393 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:00.864728 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63c6b9-6b11-4466-bd5b-cb676d191393 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:00.865208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf63c6b9-6b11-4466-bd5b-cb676d191393 None admin] PATH_INFO: `/v3/users/1e3486cfff3c4193b5a188a9bc5a211d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:00.869106 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 136/254] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:00.871177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4213d-f114-4609-a078-aa7ff5394b57 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:00.871667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4213d-f114-4609-a078-aa7ff5394b57 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:00.872154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4213d-f114-4609-a078-aa7ff5394b57 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:00.874585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 137/255] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:00.875910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4213d-f114-4609-a078-aa7ff5394b57 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:00.877839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63c6b9-6b11-4466-bd5b-cb676d191393 None admin] RBAC: Authorizing `identity:get_user(user_id=1e3486cfff3c4193b5a188a9bc5a211d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:00.880541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf63c6b9-6b11-4466-bd5b-cb676d191393 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:00.883627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 119/256] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:08:00 2022] GET /identity/v3/users/1e3486cfff3c4193b5a188a9bc5a211d => generated 297 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:00.885696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597d553-8d1a-48ee-b01c-112cc61469cd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:00.886967 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597d553-8d1a-48ee-b01c-112cc61469cd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:00.887473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597d553-8d1a-48ee-b01c-112cc61469cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:00.911221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:00.931907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7597d553-8d1a-48ee-b01c-112cc61469cd None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:00.933143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7597d553-8d1a-48ee-b01c-112cc61469cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:01.108776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4213d-f114-4609-a078-aa7ff5394b57 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:01.114892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 120/257] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:01.115776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597d553-8d1a-48ee-b01c-112cc61469cd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:01.136604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:01.198633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc752a-7354-4f83-b101-97dc4f9d21b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:01.199286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc752a-7354-4f83-b101-97dc4f9d21b2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:01.199563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc752a-7354-4f83-b101-97dc4f9d21b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:01.199828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bc752a-7354-4f83-b101-97dc4f9d21b2 None admin] PATH_INFO: `/v3/users/450aee69bbe44df982aa0f8ea6872283` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:01.201568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc752a-7354-4f83-b101-97dc4f9d21b2 None admin] RBAC: Authorizing `identity:get_user(user_id=450aee69bbe44df982aa0f8ea6872283)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:01.207486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bc752a-7354-4f83-b101-97dc4f9d21b2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:01.209211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 138/258] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:08:00 2022] GET /identity/v3/users/450aee69bbe44df982aa0f8ea6872283 => generated 283 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:01.228387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc752a-7354-4f83-b101-97dc4f9d21b2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:01.286865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597d553-8d1a-48ee-b01c-112cc61469cd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:01.407445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af688c5f-b71b-4e5e-a679-cbf57ac5c444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:01.408616 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af688c5f-b71b-4e5e-a679-cbf57ac5c444 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:01.409048 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af688c5f-b71b-4e5e-a679-cbf57ac5c444 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:01.409556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af688c5f-b71b-4e5e-a679-cbf57ac5c444 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:01.414736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af688c5f-b71b-4e5e-a679-cbf57ac5c444 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:01.423664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af688c5f-b71b-4e5e-a679-cbf57ac5c444 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:01.426948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 121/259] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:01 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:01.433548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af688c5f-b71b-4e5e-a679-cbf57ac5c444 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:01.441847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:01.548388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bc752a-7354-4f83-b101-97dc4f9d21b2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:01.615455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:01.629382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af688c5f-b71b-4e5e-a679-cbf57ac5c444 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:01.656941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf53e57-5c79-4dd1-8560-a270bf9d04a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:01.658085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf53e57-5c79-4dd1-8560-a270bf9d04a0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:01.658553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf53e57-5c79-4dd1-8560-a270bf9d04a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:01.659039 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf53e57-5c79-4dd1-8560-a270bf9d04a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:01.673319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8251c-f03f-4639-9cb2-4eda98c41d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:01.674007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8251c-f03f-4639-9cb2-4eda98c41d6f None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:01.674318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8251c-f03f-4639-9cb2-4eda98c41d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:01.674602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8251c-f03f-4639-9cb2-4eda98c41d6f None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9/users/1e3486cfff3c4193b5a188a9bc5a211d/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:01.682491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8251c-f03f-4639-9cb2-4eda98c41d6f None admin] RBAC: Authorizing `identity:create_grant(project_id=6753f2523be140a3a48348df431cf0d9, user_id=1e3486cfff3c4193b5a188a9bc5a211d, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:01.684340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8251c-f03f-4639-9cb2-4eda98c41d6f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:01.706718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 122/260] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:08:01 2022] PUT /identity/v3/projects/6753f2523be140a3a48348df431cf0d9/users/1e3486cfff3c4193b5a188a9bc5a211d/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:08:01.707906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8251c-f03f-4639-9cb2-4eda98c41d6f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:01.841302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf53e57-5c79-4dd1-8560-a270bf9d04a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:01.842993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf53e57-5c79-4dd1-8560-a270bf9d04a0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:01.994415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8251c-f03f-4639-9cb2-4eda98c41d6f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:01.998888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:02.079266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 139/261] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 851 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 16:08:02.084687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf53e57-5c79-4dd1-8560-a270bf9d04a0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:02.111173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697c35f-b168-4d98-8959-4d162bb60891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:02.112406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697c35f-b168-4d98-8959-4d162bb60891 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:02.112895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697c35f-b168-4d98-8959-4d162bb60891 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:02.113398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697c35f-b168-4d98-8959-4d162bb60891 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:02.134763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:02.145940 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697c35f-b168-4d98-8959-4d162bb60891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-f697c35f-b168-4d98-8959-4d162bb60891 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:02.165472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:02.167370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:02.171335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 123/262] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:08:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 460 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:02.177272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697c35f-b168-4d98-8959-4d162bb60891 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:02.375203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697c35f-b168-4d98-8959-4d162bb60891 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:02.380674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf53e57-5c79-4dd1-8560-a270bf9d04a0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:02.496736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e3b2e-1ca7-4edc-984b-deb9da25d3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:02.497753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e3b2e-1ca7-4edc-984b-deb9da25d3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:02.498269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e3b2e-1ca7-4edc-984b-deb9da25d3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:02.498769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e3b2e-1ca7-4edc-984b-deb9da25d3a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:02.500238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e3b2e-1ca7-4edc-984b-deb9da25d3a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:02.500761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e3b2e-1ca7-4edc-984b-deb9da25d3a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:02.503552 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e3b2e-1ca7-4edc-984b-deb9da25d3a6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:02.503857 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165a718-ced6-42de-8a9b-5a6bdaa1b019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:02.504666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165a718-ced6-42de-8a9b-5a6bdaa1b019 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:02.504920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165a718-ced6-42de-8a9b-5a6bdaa1b019 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:02.505184 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165a718-ced6-42de-8a9b-5a6bdaa1b019 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:02.506266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165a718-ced6-42de-8a9b-5a6bdaa1b019 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:02.507769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165a718-ced6-42de-8a9b-5a6bdaa1b019 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:02.509490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 140/263] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:02 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:02.515643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165a718-ced6-42de-8a9b-5a6bdaa1b019 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:02.532830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 124/264] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:02.542841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e3b2e-1ca7-4edc-984b-deb9da25d3a6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:02.595347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:02.706530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:02.754151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e3b2e-1ca7-4edc-984b-deb9da25d3a6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:02.794766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7699b-f7bd-4737-b606-75b88d059ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:02.795427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7699b-f7bd-4737-b606-75b88d059ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:02.795716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7699b-f7bd-4737-b606-75b88d059ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:02.795995 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7699b-f7bd-4737-b606-75b88d059ea6 None admin] PATH_INFO: `/v3/users/450aee69bbe44df982aa0f8ea6872283` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:02.797257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7699b-f7bd-4737-b606-75b88d059ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=450aee69bbe44df982aa0f8ea6872283)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:02.800522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165a718-ced6-42de-8a9b-5a6bdaa1b019 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:02.802928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7699b-f7bd-4737-b606-75b88d059ea6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:02.805170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 125/265] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:08:02 2022] GET /identity/v3/users/450aee69bbe44df982aa0f8ea6872283 => generated 283 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:02.811492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7699b-f7bd-4737-b606-75b88d059ea6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:02.930148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7699b-f7bd-4737-b606-75b88d059ea6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:02.947241 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547aced-c936-4194-8b1e-3c6cbfaa52c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:02.948723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547aced-c936-4194-8b1e-3c6cbfaa52c7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:02.949494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547aced-c936-4194-8b1e-3c6cbfaa52c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:02.950318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547aced-c936-4194-8b1e-3c6cbfaa52c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:02.952824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547aced-c936-4194-8b1e-3c6cbfaa52c7 None admin] RBAC: Adding query filter params (role.id=aa320f864d914f65bebe5463994ff536, scope.project.id=6753f2523be140a3a48348df431cf0d9, user.id=450aee69bbe44df982aa0f8ea6872283) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:02.953729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547aced-c936-4194-8b1e-3c6cbfaa52c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:02.961057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547aced-c936-4194-8b1e-3c6cbfaa52c7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:02.962416 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:02.974875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddce1ce-d6fb-4757-9159-6035645db4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:02.976927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddce1ce-d6fb-4757-9159-6035645db4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:02.977305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddce1ce-d6fb-4757-9159-6035645db4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:02.977860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddce1ce-d6fb-4757-9159-6035645db4e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:02.986222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 141/266] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:08:02 2022] GET /identity/v3/role_assignments?user.id=450aee69bbe44df982aa0f8ea6872283&scope.project.id=6753f2523be140a3a48348df431cf0d9&role.id=aa320f864d914f65bebe5463994ff536 => generated 608 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:03.035958 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ded1a-cdc6-46f8-80f7-8fa6f9f80a3e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:03.036897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ded1a-cdc6-46f8-80f7-8fa6f9f80a3e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:03.037681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ded1a-cdc6-46f8-80f7-8fa6f9f80a3e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:03.046781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 142/267] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:03 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:08:03.061014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e247a85-78d6-4d4b-b06a-17e724e5b00f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:03.062217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e247a85-78d6-4d4b-b06a-17e724e5b00f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:03.063100 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e247a85-78d6-4d4b-b06a-17e724e5b00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:03.095616 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddce1ce-d6fb-4757-9159-6035645db4e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:03.097144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddce1ce-d6fb-4757-9159-6035645db4e9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:03.131884 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7e247a85-78d6-4d4b-b06a-17e724e5b00f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:03.134496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7e247a85-78d6-4d4b-b06a-17e724e5b00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:03.231801 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 126/268] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 16:08:03.239306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddce1ce-d6fb-4757-9159-6035645db4e9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:03.291300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:03.371716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddce1ce-d6fb-4757-9159-6035645db4e9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:03.391852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 143/269] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:03.392668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c39182-6aa2-4d0b-8d85-d02f7786bee9 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:03.393141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c39182-6aa2-4d0b-8d85-d02f7786bee9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:03.393612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c39182-6aa2-4d0b-8d85-d02f7786bee9 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:03.397325 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 144/270] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:03.404874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547c192-6a1b-476b-9b1f-a23fd07c4320 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:03.405455 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547c192-6a1b-476b-9b1f-a23fd07c4320 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:03.406005 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547c192-6a1b-476b-9b1f-a23fd07c4320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:03.446352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c547c192-6a1b-476b-9b1f-a23fd07c4320 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:03.448300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c547c192-6a1b-476b-9b1f-a23fd07c4320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:03.467766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbabea4-835d-4aeb-9af6-20322d1cfc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:03.468440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbabea4-835d-4aeb-9af6-20322d1cfc02 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:03.468719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbabea4-835d-4aeb-9af6-20322d1cfc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:03.468999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbabea4-835d-4aeb-9af6-20322d1cfc02 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:03.470311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbabea4-835d-4aeb-9af6-20322d1cfc02 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:03.471807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbabea4-835d-4aeb-9af6-20322d1cfc02 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:03.473918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 127/271] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:03 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:03.479833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbabea4-835d-4aeb-9af6-20322d1cfc02 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:03.528775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:03.615422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 145/272] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:03.660484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:03.671242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbabea4-835d-4aeb-9af6-20322d1cfc02 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:03.752036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ac56a-880e-44fd-bff0-069ac2ff3117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:03.753298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ac56a-880e-44fd-bff0-069ac2ff3117 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:03.753981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ac56a-880e-44fd-bff0-069ac2ff3117 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:03.754681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ac56a-880e-44fd-bff0-069ac2ff3117 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:03.759241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ac56a-880e-44fd-bff0-069ac2ff3117 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=6753f2523be140a3a48348df431cf0d9, user.id=450aee69bbe44df982aa0f8ea6872283) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:03.760024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ac56a-880e-44fd-bff0-069ac2ff3117 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:03.763012 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ac56a-880e-44fd-bff0-069ac2ff3117 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:03.779943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 128/273] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:08:03 2022] GET /identity/v3/role_assignments?user.id=450aee69bbe44df982aa0f8ea6872283&scope.project.id=6753f2523be140a3a48348df431cf0d9&role.id=459d01a218314543a7a8329e88b78331 => generated 608 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:03.821982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547c192-6a1b-476b-9b1f-a23fd07c4320 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:03.962153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547c192-6a1b-476b-9b1f-a23fd07c4320 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:04.025513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b98e1-1832-416d-aed4-9d006937ee0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:04.026096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b98e1-1832-416d-aed4-9d006937ee0f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:04.026386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b98e1-1832-416d-aed4-9d006937ee0f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:04.026668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b98e1-1832-416d-aed4-9d006937ee0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:04.041449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b98e1-1832-416d-aed4-9d006937ee0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-792b98e1-1832-416d-aed4-9d006937ee0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:04.047076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:04.048688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:04.050256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 146/274] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:04.055082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ac56a-880e-44fd-bff0-069ac2ff3117 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:04.109875 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b98e1-1832-416d-aed4-9d006937ee0f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:04.144943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:04.280684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ac56a-880e-44fd-bff0-069ac2ff3117 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:04.312853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b98e1-1832-416d-aed4-9d006937ee0f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:04.350335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133e145-dffe-41f9-9eb8-714a6eec07a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:04.351338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133e145-dffe-41f9-9eb8-714a6eec07a4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:04.351827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133e145-dffe-41f9-9eb8-714a6eec07a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:04.352308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133e145-dffe-41f9-9eb8-714a6eec07a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:04.353524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133e145-dffe-41f9-9eb8-714a6eec07a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:04.354088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133e145-dffe-41f9-9eb8-714a6eec07a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:04.356829 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133e145-dffe-41f9-9eb8-714a6eec07a4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:04.376353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 129/275] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:04.384703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133e145-dffe-41f9-9eb8-714a6eec07a4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:04.391042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a02f97-7c2b-422d-9892-ce3574ece987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:04.393584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a02f97-7c2b-422d-9892-ce3574ece987 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:04.394024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a02f97-7c2b-422d-9892-ce3574ece987 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:04.394535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a02f97-7c2b-422d-9892-ce3574ece987 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:04.396704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a02f97-7c2b-422d-9892-ce3574ece987 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:04.399671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a02f97-7c2b-422d-9892-ce3574ece987 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:04.404617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 147/276] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:04.416540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a02f97-7c2b-422d-9892-ce3574ece987 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:04.424758 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:04.524745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133e145-dffe-41f9-9eb8-714a6eec07a4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:04.565668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4a2ec-8277-4d25-bff4-cb1103d50790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:04.566278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4a2ec-8277-4d25-bff4-cb1103d50790 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:04.566550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4a2ec-8277-4d25-bff4-cb1103d50790 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:04.566826 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4a2ec-8277-4d25-bff4-cb1103d50790 None admin] PATH_INFO: `/v3/users/918584c1108743d29c1bc83297fb7a5d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:04.588439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4a2ec-8277-4d25-bff4-cb1103d50790 None admin] RBAC: Authorizing `identity:get_user(user_id=918584c1108743d29c1bc83297fb7a5d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:04.590109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4a2ec-8277-4d25-bff4-cb1103d50790 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:04.598381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 130/277] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:08:04 2022] GET /identity/v3/users/918584c1108743d29c1bc83297fb7a5d => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:04.604045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4a2ec-8277-4d25-bff4-cb1103d50790 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:04.620366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:04.634477 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a02f97-7c2b-422d-9892-ce3574ece987 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:04.698811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe2d8d-e6dc-46b6-840c-3f5668ca688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:04.700341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe2d8d-e6dc-46b6-840c-3f5668ca688a None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:04.701944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe2d8d-e6dc-46b6-840c-3f5668ca688a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:04.702839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe2d8d-e6dc-46b6-840c-3f5668ca688a None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:04.704401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe2d8d-e6dc-46b6-840c-3f5668ca688a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:04.707553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe2d8d-e6dc-46b6-840c-3f5668ca688a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:04.734708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:04.751075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 148/278] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:08:04 2022] POST /identity/v3/users => generated 293 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:08:04.767737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4a2ec-8277-4d25-bff4-cb1103d50790 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:04.772961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca92222-95d7-476f-9147-c3683e20f024 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:04.773964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca92222-95d7-476f-9147-c3683e20f024 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:04.774823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca92222-95d7-476f-9147-c3683e20f024 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:04.778251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 149/279] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:04.786520 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84a35d-51cc-4b84-8bee-a83e350b9752 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:04.787521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84a35d-51cc-4b84-8bee-a83e350b9752 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:04.789267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f84a35d-51cc-4b84-8bee-a83e350b9752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:04.822236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0f84a35d-51cc-4b84-8bee-a83e350b9752 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:04.823604 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0f84a35d-51cc-4b84-8bee-a83e350b9752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:04.825338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657fe485-5975-497d-9111-ce8ffa04f7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:04.829736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fe485-5975-497d-9111-ce8ffa04f7bb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:04.830615 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fe485-5975-497d-9111-ce8ffa04f7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:04.831263 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657fe485-5975-497d-9111-ce8ffa04f7bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:04.832894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657fe485-5975-497d-9111-ce8ffa04f7bb None admin] RBAC: Adding query filter params (role.id=20378c15bf524a4c960b49d2c53cf6b2, user.id=918584c1108743d29c1bc83297fb7a5d, scope.system=all) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:04.833679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657fe485-5975-497d-9111-ce8ffa04f7bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:04.838882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657fe485-5975-497d-9111-ce8ffa04f7bb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:04.862092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 131/280] 10.222.0.64 () {56 vars in 1308 bytes} [Wed Sep 21 16:08:04 2022] GET /identity/v3/role_assignments?user.id=918584c1108743d29c1bc83297fb7a5d&scope.system=all&role.id=20378c15bf524a4c960b49d2c53cf6b2 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:04.928326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 150/281] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:05.213137 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657fe485-5975-497d-9111-ce8ffa04f7bb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:05.332502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d45b74-5c26-4b62-ad0d-feab2ccd4b3b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:05.332790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d45b74-5c26-4b62-ad0d-feab2ccd4b3b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:05.333061 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d45b74-5c26-4b62-ad0d-feab2ccd4b3b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:05.334837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 151/282] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:05.341904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366cef6-06eb-42eb-a417-d17e1a377b5d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:05.342356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366cef6-06eb-42eb-a417-d17e1a377b5d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:05.342496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366cef6-06eb-42eb-a417-d17e1a377b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:05.371407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6366cef6-06eb-42eb-a417-d17e1a377b5d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:05.373216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6366cef6-06eb-42eb-a417-d17e1a377b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:05.379721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657fe485-5975-497d-9111-ce8ffa04f7bb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:05.445214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2a4bc-6223-48ba-88ea-1271ad55af2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:05.445880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2a4bc-6223-48ba-88ea-1271ad55af2a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:05.446203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2a4bc-6223-48ba-88ea-1271ad55af2a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:05.446476 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2a4bc-6223-48ba-88ea-1271ad55af2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:05.466410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2a4bc-6223-48ba-88ea-1271ad55af2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c0d2a4bc-6223-48ba-88ea-1271ad55af2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:05.471445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:05.473154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:05.474767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 132/283] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:05.488135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2a4bc-6223-48ba-88ea-1271ad55af2a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:05.489481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:05.548014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 152/284] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:08:05.652530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2a4bc-6223-48ba-88ea-1271ad55af2a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:05.693794 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a33c27-7cd0-466d-bea6-8b642f9eba95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:05.694535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a33c27-7cd0-466d-bea6-8b642f9eba95 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:05.694816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a33c27-7cd0-466d-bea6-8b642f9eba95 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:05.695082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a33c27-7cd0-466d-bea6-8b642f9eba95 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:05.695782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a33c27-7cd0-466d-bea6-8b642f9eba95 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:05.696089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a33c27-7cd0-466d-bea6-8b642f9eba95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:05.697529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a33c27-7cd0-466d-bea6-8b642f9eba95 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:05.708808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 133/285] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:05.714729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366cef6-06eb-42eb-a417-d17e1a377b5d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:05.730541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:05.865209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366cef6-06eb-42eb-a417-d17e1a377b5d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:05.917265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83275d-1613-49be-ad1c-430e3c5512ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:05.917989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83275d-1613-49be-ad1c-430e3c5512ff None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:05.918295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83275d-1613-49be-ad1c-430e3c5512ff None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:05.918566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83275d-1613-49be-ad1c-430e3c5512ff None admin] PATH_INFO: `/v3/users/f29809ecb6ed495c9e4b687f2afaa41f` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:05.922889 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83275d-1613-49be-ad1c-430e3c5512ff None admin] RBAC: Authorizing `identity:get_user(user_id=f29809ecb6ed495c9e4b687f2afaa41f)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:05.924370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83275d-1613-49be-ad1c-430e3c5512ff None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:05.926084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 153/286] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:08:05 2022] GET /identity/v3/users/f29809ecb6ed495c9e4b687f2afaa41f => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:05.932001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a33c27-7cd0-466d-bea6-8b642f9eba95 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:06.021021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83275d-1613-49be-ad1c-430e3c5512ff None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:06.023730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:06.074208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a33c27-7cd0-466d-bea6-8b642f9eba95 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:06.113393 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032757e5-00d1-4403-9373-1f70951e3987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:06.114077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032757e5-00d1-4403-9373-1f70951e3987 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:06.114356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032757e5-00d1-4403-9373-1f70951e3987 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:06.114618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032757e5-00d1-4403-9373-1f70951e3987 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:06.115849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032757e5-00d1-4403-9373-1f70951e3987 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:06.118553 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032757e5-00d1-4403-9373-1f70951e3987 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:06.120604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 134/287] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:05 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:06.126695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032757e5-00d1-4403-9373-1f70951e3987 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:06.240122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032757e5-00d1-4403-9373-1f70951e3987 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:06.281797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13403b3-007a-4707-9827-95d5570440e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:06.282426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13403b3-007a-4707-9827-95d5570440e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:06.282707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13403b3-007a-4707-9827-95d5570440e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:06.282992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13403b3-007a-4707-9827-95d5570440e5 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9/users/f29809ecb6ed495c9e4b687f2afaa41f/roles/bbbfb342bd244610906d836359f61fcf` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:06.285231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13403b3-007a-4707-9827-95d5570440e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6753f2523be140a3a48348df431cf0d9, user_id=f29809ecb6ed495c9e4b687f2afaa41f, role_id=bbbfb342bd244610906d836359f61fcf)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:06.286791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13403b3-007a-4707-9827-95d5570440e5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:06.290539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:06.301252 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:06.312889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 135/288] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:08:06 2022] PUT /identity/v3/projects/6753f2523be140a3a48348df431cf0d9/users/f29809ecb6ed495c9e4b687f2afaa41f/roles/bbbfb342bd244610906d836359f61fcf => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:08:06.380660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83275d-1613-49be-ad1c-430e3c5512ff None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:06.476491 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a89c23-e084-4539-868c-e32305786792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:06.477575 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a89c23-e084-4539-868c-e32305786792 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:06.478076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a89c23-e084-4539-868c-e32305786792 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:06.478570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a89c23-e084-4539-868c-e32305786792 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:06.519300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a89c23-e084-4539-868c-e32305786792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-b7a89c23-e084-4539-868c-e32305786792 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:06.528915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:06.530716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:06.532436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:06.532436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:06.532436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:06.532436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:06.532436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:06.532436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:06.532436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:06.532436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:06.532436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:06.532436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:06.532436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:06.532436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:06.533657 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 154/289] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 511 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:06.540298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13403b3-007a-4707-9827-95d5570440e5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:06.702772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13403b3-007a-4707-9827-95d5570440e5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:06.769907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd1399-8f08-4a2e-9b88-7d92606d6bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:06.770575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd1399-8f08-4a2e-9b88-7d92606d6bdf None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:06.770859 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd1399-8f08-4a2e-9b88-7d92606d6bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:06.774086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd1399-8f08-4a2e-9b88-7d92606d6bdf None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:06.774086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd1399-8f08-4a2e-9b88-7d92606d6bdf None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:06.774086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd1399-8f08-4a2e-9b88-7d92606d6bdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:06.774086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd1399-8f08-4a2e-9b88-7d92606d6bdf None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:06.788489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 136/290] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:06.797689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a89c23-e084-4539-868c-e32305786792 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:07.051211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a89c23-e084-4539-868c-e32305786792 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:07.125299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67685fa5-95eb-44e7-9119-7f5e823d2e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:07.125948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67685fa5-95eb-44e7-9119-7f5e823d2e12 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:07.126244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67685fa5-95eb-44e7-9119-7f5e823d2e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:07.126512 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67685fa5-95eb-44e7-9119-7f5e823d2e12 None admin] PATH_INFO: `/v3/users/1e3486cfff3c4193b5a188a9bc5a211d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:07.127694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67685fa5-95eb-44e7-9119-7f5e823d2e12 None admin] RBAC: Authorizing `identity:get_user(user_id=1e3486cfff3c4193b5a188a9bc5a211d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:07.129191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67685fa5-95eb-44e7-9119-7f5e823d2e12 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:07.130786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 155/291] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:08:06 2022] GET /identity/v3/users/1e3486cfff3c4193b5a188a9bc5a211d => generated 297 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:07.139687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd1399-8f08-4a2e-9b88-7d92606d6bdf None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:07.232255 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:07.298391 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd1399-8f08-4a2e-9b88-7d92606d6bdf None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:07.342577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160e216-aab1-44aa-8a40-d07a32406cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:07.343100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160e216-aab1-44aa-8a40-d07a32406cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:07.343371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160e216-aab1-44aa-8a40-d07a32406cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:07.343638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0160e216-aab1-44aa-8a40-d07a32406cb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:07.440576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0160e216-aab1-44aa-8a40-d07a32406cb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:07.442095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0160e216-aab1-44aa-8a40-d07a32406cb3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:07.522096 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:07.582616 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 137/292] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:07.586909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67685fa5-95eb-44e7-9119-7f5e823d2e12 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:07.599395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:07.741528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67685fa5-95eb-44e7-9119-7f5e823d2e12 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:07.812568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eed253-1a9f-4812-874e-8c6330ff45f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:07.813204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eed253-1a9f-4812-874e-8c6330ff45f1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:07.813496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eed253-1a9f-4812-874e-8c6330ff45f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:07.813858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eed253-1a9f-4812-874e-8c6330ff45f1 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:07.819032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eed253-1a9f-4812-874e-8c6330ff45f1 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:07.820536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eed253-1a9f-4812-874e-8c6330ff45f1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:07.822862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 156/293] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:07 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:07.832399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160e216-aab1-44aa-8a40-d07a32406cb3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:07.976161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0160e216-aab1-44aa-8a40-d07a32406cb3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:08.019047 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561ba9d-5999-46d2-9719-bafdacb3a3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:08.019679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561ba9d-5999-46d2-9719-bafdacb3a3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:08.019956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561ba9d-5999-46d2-9719-bafdacb3a3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:08.020224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561ba9d-5999-46d2-9719-bafdacb3a3e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:08.020999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561ba9d-5999-46d2-9719-bafdacb3a3e2 None admin] RBAC: Adding query filter params (role.id=20378c15bf524a4c960b49d2c53cf6b2, scope.project.id=6753f2523be140a3a48348df431cf0d9, user.id=1e3486cfff3c4193b5a188a9bc5a211d) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:08.021308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561ba9d-5999-46d2-9719-bafdacb3a3e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:08.022837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561ba9d-5999-46d2-9719-bafdacb3a3e2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:08.031104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 138/294] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:08:07 2022] GET /identity/v3/role_assignments?user.id=1e3486cfff3c4193b5a188a9bc5a211d&scope.project.id=6753f2523be140a3a48348df431cf0d9&role.id=20378c15bf524a4c960b49d2c53cf6b2 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:08.160821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d14037-455a-4ae4-aa51-60815f5a071a None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:08.161265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d14037-455a-4ae4-aa51-60815f5a071a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:08.161706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d14037-455a-4ae4-aa51-60815f5a071a None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:08.164719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 157/295] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:08.174169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ddc9d-2d31-4b33-8d76-a583bce63cc4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:08.174732 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ddc9d-2d31-4b33-8d76-a583bce63cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:08.174996 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ddc9d-2d31-4b33-8d76-a583bce63cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:08.205061 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-784ddc9d-2d31-4b33-8d76-a583bce63cc4 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:08.207054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-784ddc9d-2d31-4b33-8d76-a583bce63cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:08.233509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:08.346761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 139/296] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:08.567235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcce24c-3936-4941-b832-6bca8e24e9a8 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:08.568141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcce24c-3936-4941-b832-6bca8e24e9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:08.568890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcce24c-3936-4941-b832-6bca8e24e9a8 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:08.572211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 158/297] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:08.580512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacedfa-a25f-42a4-bc0a-49538ce90a30 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:08.581433 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacedfa-a25f-42a4-bc0a-49538ce90a30 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:08.582261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dacedfa-a25f-42a4-bc0a-49538ce90a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:08.619091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5dacedfa-a25f-42a4-bc0a-49538ce90a30 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:08.621622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5dacedfa-a25f-42a4-bc0a-49538ce90a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:08.773817 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 140/298] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:08.804947 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcce24c-3936-4941-b832-6bca8e24e9a8 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:08.839708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4c0ea-7528-4f4b-9ae7-b09aea6b2230 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:08.840418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4c0ea-7528-4f4b-9ae7-b09aea6b2230 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:08.840932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4c0ea-7528-4f4b-9ae7-b09aea6b2230 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:08.843228 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 141/299] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:08:08.849221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c0c78-1e3e-4fd2-8898-4301feaaeb7c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:08.849539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c0c78-1e3e-4fd2-8898-4301feaaeb7c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:08.849787 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c0c78-1e3e-4fd2-8898-4301feaaeb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:08.869913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cf2c0c78-1e3e-4fd2-8898-4301feaaeb7c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:08.871476 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cf2c0c78-1e3e-4fd2-8898-4301feaaeb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:08.947131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 142/300] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:08:08.970424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:09.006063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcce24c-3936-4941-b832-6bca8e24e9a8 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:09.059264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c0c78-1e3e-4fd2-8898-4301feaaeb7c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:09.072790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97491f70-28f1-412b-a31e-978fad7fae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:09.073457 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:09.073872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97491f70-28f1-412b-a31e-978fad7fae6c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:09.074399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97491f70-28f1-412b-a31e-978fad7fae6c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:09.074873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97491f70-28f1-412b-a31e-978fad7fae6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:09.077814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97491f70-28f1-412b-a31e-978fad7fae6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:09.080618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97491f70-28f1-412b-a31e-978fad7fae6c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:09.083870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 159/301] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:09.091381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97491f70-28f1-412b-a31e-978fad7fae6c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:09.179622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c0c78-1e3e-4fd2-8898-4301feaaeb7c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:09.220331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa41e7f-ee70-4f83-88de-b7c28f8ba6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:09.221330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa41e7f-ee70-4f83-88de-b7c28f8ba6eb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:09.221790 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa41e7f-ee70-4f83-88de-b7c28f8ba6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:09.222290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa41e7f-ee70-4f83-88de-b7c28f8ba6eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:09.234420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa41e7f-ee70-4f83-88de-b7c28f8ba6eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0aa41e7f-ee70-4f83-88de-b7c28f8ba6eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:09.240435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:09.241483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:09.242300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 143/302] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:09.243284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3822f9-d139-48f4-ada2-e4d09b2f5d7e None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:09.243576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3822f9-d139-48f4-ada2-e4d09b2f5d7e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:09.243853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3822f9-d139-48f4-ada2-e4d09b2f5d7e None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:09.245670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 144/303] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:09.246506 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3822f9-d139-48f4-ada2-e4d09b2f5d7e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:09.259278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97491f70-28f1-412b-a31e-978fad7fae6c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:09.295912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a9c2dd-dfaf-4b49-9bc8-59f340cd7c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:09.296542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9c2dd-dfaf-4b49-9bc8-59f340cd7c19 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:09.296828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9c2dd-dfaf-4b49-9bc8-59f340cd7c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:09.297103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a9c2dd-dfaf-4b49-9bc8-59f340cd7c19 None admin] PATH_INFO: `/v3/groups` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:09.297755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9c2dd-dfaf-4b49-9bc8-59f340cd7c19 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:09.299245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a9c2dd-dfaf-4b49-9bc8-59f340cd7c19 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:09.316355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 160/304] 10.222.0.64 () {60 vars in 1134 bytes} [Wed Sep 21 16:08:09 2022] POST /identity/v3/groups => generated 229 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:08:09.317546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9244c-2428-4e3b-9cab-d97bca919ecb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:09.317838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9244c-2428-4e3b-9cab-d97bca919ecb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:09.318143 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9244c-2428-4e3b-9cab-d97bca919ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:09.324689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:09.346021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-00a9244c-2428-4e3b-9cab-d97bca919ecb None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:09.347354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-00a9244c-2428-4e3b-9cab-d97bca919ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:09.352045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:09.424262 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3822f9-d139-48f4-ada2-e4d09b2f5d7e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:09.443534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 161/305] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:09.445026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9244c-2428-4e3b-9cab-d97bca919ecb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:09.544761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dda26-5bd8-4f20-a72c-56453f745640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:09.545949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dda26-5bd8-4f20-a72c-56453f745640 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:09.546470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dda26-5bd8-4f20-a72c-56453f745640 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:09.547090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94dda26-5bd8-4f20-a72c-56453f745640 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:09.570197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94dda26-5bd8-4f20-a72c-56453f745640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b94dda26-5bd8-4f20-a72c-56453f745640 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:09.577651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:09.579540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:09.580920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 145/306] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:09.588983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dda26-5bd8-4f20-a72c-56453f745640 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:09.650969 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9244c-2428-4e3b-9cab-d97bca919ecb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:09.693237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:09.701904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e6a39-3a69-48c7-88c6-5ffb1004cf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:09.703138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e6a39-3a69-48c7-88c6-5ffb1004cf4b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:09.703534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e6a39-3a69-48c7-88c6-5ffb1004cf4b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:09.703914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e6a39-3a69-48c7-88c6-5ffb1004cf4b None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:09.705222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e6a39-3a69-48c7-88c6-5ffb1004cf4b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:09.705646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e6a39-3a69-48c7-88c6-5ffb1004cf4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:09.707255 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e6a39-3a69-48c7-88c6-5ffb1004cf4b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:09.720402 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 162/307] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:09.726923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e6a39-3a69-48c7-88c6-5ffb1004cf4b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:09.800001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94dda26-5bd8-4f20-a72c-56453f745640 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:09.831150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:09.840919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26730044-e8dd-49b9-989b-c7d316c18758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:09.841683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26730044-e8dd-49b9-989b-c7d316c18758 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:09.842099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26730044-e8dd-49b9-989b-c7d316c18758 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:09.842484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26730044-e8dd-49b9-989b-c7d316c18758 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:09.843845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26730044-e8dd-49b9-989b-c7d316c18758 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:09.843845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26730044-e8dd-49b9-989b-c7d316c18758 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:09.845156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26730044-e8dd-49b9-989b-c7d316c18758 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:09.862080 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 146/308] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:09.866778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26730044-e8dd-49b9-989b-c7d316c18758 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:09.925050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e6a39-3a69-48c7-88c6-5ffb1004cf4b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:09.985377 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02755f65-e743-4e71-9f5f-e7eeff384934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:09.986464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02755f65-e743-4e71-9f5f-e7eeff384934 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:09.986895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02755f65-e743-4e71-9f5f-e7eeff384934 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:09.987612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02755f65-e743-4e71-9f5f-e7eeff384934 None admin] PATH_INFO: `/v3/users/f88f34f0da6c4c68b99065891776fa4c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:10.005800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02755f65-e743-4e71-9f5f-e7eeff384934 None admin] RBAC: Authorizing `identity:get_user(user_id=f88f34f0da6c4c68b99065891776fa4c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:10.007329 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02755f65-e743-4e71-9f5f-e7eeff384934 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:10.009365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 163/309] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:08:09 2022] GET /identity/v3/users/f88f34f0da6c4c68b99065891776fa4c => generated 293 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:10.010327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26730044-e8dd-49b9-989b-c7d316c18758 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:10.015010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02755f65-e743-4e71-9f5f-e7eeff384934 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:10.062239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4375617-d33a-4fba-a6d5-c95eb4611664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:10.063208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4375617-d33a-4fba-a6d5-c95eb4611664 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:10.063622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4375617-d33a-4fba-a6d5-c95eb4611664 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:10.063901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4375617-d33a-4fba-a6d5-c95eb4611664 None admin] PATH_INFO: `/v3/users/918584c1108743d29c1bc83297fb7a5d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:10.065166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4375617-d33a-4fba-a6d5-c95eb4611664 None admin] RBAC: Authorizing `identity:get_user(user_id=918584c1108743d29c1bc83297fb7a5d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:10.066694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4375617-d33a-4fba-a6d5-c95eb4611664 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:10.069854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 147/310] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:08:09 2022] GET /identity/v3/users/918584c1108743d29c1bc83297fb7a5d => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:10.075606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4375617-d33a-4fba-a6d5-c95eb4611664 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:10.103306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:10.169281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02755f65-e743-4e71-9f5f-e7eeff384934 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:10.177691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:10.215318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3dae64-7e08-460a-9e43-f1c9336ff27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:10.216757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3dae64-7e08-460a-9e43-f1c9336ff27c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:10.217166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3dae64-7e08-460a-9e43-f1c9336ff27c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:10.217566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3dae64-7e08-460a-9e43-f1c9336ff27c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:10.231239 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3dae64-7e08-460a-9e43-f1c9336ff27c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-7c3dae64-7e08-460a-9e43-f1c9336ff27c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:10.237093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:10.238887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 164/311] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:10.241859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3dae64-7e08-460a-9e43-f1c9336ff27c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:10.263369 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4375617-d33a-4fba-a6d5-c95eb4611664 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:10.307922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de92a36-62b9-49fa-b73d-62a3297651ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:10.308578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de92a36-62b9-49fa-b73d-62a3297651ba None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:10.308866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de92a36-62b9-49fa-b73d-62a3297651ba None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:10.309144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de92a36-62b9-49fa-b73d-62a3297651ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:10.309902 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de92a36-62b9-49fa-b73d-62a3297651ba None admin] RBAC: Adding query filter params (role.id=bbbfb342bd244610906d836359f61fcf, user.id=f88f34f0da6c4c68b99065891776fa4c, scope.system=all) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:10.310244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de92a36-62b9-49fa-b73d-62a3297651ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:10.311696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de92a36-62b9-49fa-b73d-62a3297651ba None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:10.318077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 148/312] 10.222.0.64 () {56 vars in 1308 bytes} [Wed Sep 21 16:08:10 2022] GET /identity/v3/role_assignments?user.id=f88f34f0da6c4c68b99065891776fa4c&scope.system=all&role.id=bbbfb342bd244610906d836359f61fcf => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:10.328561 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de92a36-62b9-49fa-b73d-62a3297651ba None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:10.442482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3dae64-7e08-460a-9e43-f1c9336ff27c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:10.471967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:10.528965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db08878-69e7-4dda-a02a-903a93eaa4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:10.531462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db08878-69e7-4dda-a02a-903a93eaa4f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:10.532249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db08878-69e7-4dda-a02a-903a93eaa4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:10.533587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db08878-69e7-4dda-a02a-903a93eaa4f2 None admin] PATH_INFO: `/v3/system/users/918584c1108743d29c1bc83297fb7a5d/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:10.537011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db08878-69e7-4dda-a02a-903a93eaa4f2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=918584c1108743d29c1bc83297fb7a5d, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:10.541183 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db08878-69e7-4dda-a02a-903a93eaa4f2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:10.559513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 165/313] 10.222.0.64 () {58 vars in 1334 bytes} [Wed Sep 21 16:08:10 2022] PUT /identity/v3/system/users/918584c1108743d29c1bc83297fb7a5d/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:08:10.586089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de92a36-62b9-49fa-b73d-62a3297651ba None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:10.630741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:10.645399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b881ede-5e3d-4c45-83f2-59b1a83d863c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:10.646029 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b881ede-5e3d-4c45-83f2-59b1a83d863c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:10.646329 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b881ede-5e3d-4c45-83f2-59b1a83d863c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:10.646652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b881ede-5e3d-4c45-83f2-59b1a83d863c None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:10.647411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b881ede-5e3d-4c45-83f2-59b1a83d863c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:10.647719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b881ede-5e3d-4c45-83f2-59b1a83d863c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:10.649145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b881ede-5e3d-4c45-83f2-59b1a83d863c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:10.660104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 149/314] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:10.666748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db08878-69e7-4dda-a02a-903a93eaa4f2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:10.842846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db08878-69e7-4dda-a02a-903a93eaa4f2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:10.880590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52de2f4-e0f0-4c68-aff7-d1f15adcf90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:10.881238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52de2f4-e0f0-4c68-aff7-d1f15adcf90d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:10.881518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52de2f4-e0f0-4c68-aff7-d1f15adcf90d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:10.881793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52de2f4-e0f0-4c68-aff7-d1f15adcf90d None admin] PATH_INFO: `/v3/users/f29809ecb6ed495c9e4b687f2afaa41f` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:10.882972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52de2f4-e0f0-4c68-aff7-d1f15adcf90d None admin] RBAC: Authorizing `identity:get_user(user_id=f29809ecb6ed495c9e4b687f2afaa41f)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:10.884402 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52de2f4-e0f0-4c68-aff7-d1f15adcf90d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:10.886305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 166/315] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:08:10 2022] GET /identity/v3/users/f29809ecb6ed495c9e4b687f2afaa41f => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:10.893456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b881ede-5e3d-4c45-83f2-59b1a83d863c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:11.006910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b881ede-5e3d-4c45-83f2-59b1a83d863c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:11.045712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38377eab-9caa-4cb0-ad06-f296625b658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:11.046363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38377eab-9caa-4cb0-ad06-f296625b658c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:11.046641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38377eab-9caa-4cb0-ad06-f296625b658c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:11.046979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38377eab-9caa-4cb0-ad06-f296625b658c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:11.107275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:11.125684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38377eab-9caa-4cb0-ad06-f296625b658c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:11.127189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38377eab-9caa-4cb0-ad06-f296625b658c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:11.241860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 150/316] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:11.247960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52de2f4-e0f0-4c68-aff7-d1f15adcf90d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:11.318492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:11.371730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52de2f4-e0f0-4c68-aff7-d1f15adcf90d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:11.409746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77eb230-9f60-4638-88f8-8de2519ca75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:11.410486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb230-9f60-4638-88f8-8de2519ca75a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:11.410773 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb230-9f60-4638-88f8-8de2519ca75a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:11.411028 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77eb230-9f60-4638-88f8-8de2519ca75a None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:11.412386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77eb230-9f60-4638-88f8-8de2519ca75a None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:11.413897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77eb230-9f60-4638-88f8-8de2519ca75a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:11.416018 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 167/317] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:11 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:11.423137 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38377eab-9caa-4cb0-ad06-f296625b658c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:11.525778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:11.545209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38377eab-9caa-4cb0-ad06-f296625b658c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:11.597099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222da798-c522-439f-a6e0-3897eb240005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:11.597734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222da798-c522-439f-a6e0-3897eb240005 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:11.598011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222da798-c522-439f-a6e0-3897eb240005 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:11.598321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222da798-c522-439f-a6e0-3897eb240005 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:11.599162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222da798-c522-439f-a6e0-3897eb240005 None admin] RBAC: Adding query filter params (role.id=bbbfb342bd244610906d836359f61fcf, scope.project.id=6753f2523be140a3a48348df431cf0d9, user.id=f29809ecb6ed495c9e4b687f2afaa41f) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:11.599468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222da798-c522-439f-a6e0-3897eb240005 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:11.600974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222da798-c522-439f-a6e0-3897eb240005 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:11.609608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 151/318] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:08:11 2022] GET /identity/v3/role_assignments?user.id=f29809ecb6ed495c9e4b687f2afaa41f&scope.project.id=6753f2523be140a3a48348df431cf0d9&role.id=bbbfb342bd244610906d836359f61fcf => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:12.628506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bf1d8-7067-4044-b8b8-3e057f1afc97 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:12.629476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bf1d8-7067-4044-b8b8-3e057f1afc97 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:12.630008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bf1d8-7067-4044-b8b8-3e057f1afc97 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:12.632040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 168/319] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:12.638086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1196f2-e152-433a-ac16-47aa20629372 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:12.638617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1196f2-e152-433a-ac16-47aa20629372 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:12.638705 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1196f2-e152-433a-ac16-47aa20629372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:12.659015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0a1196f2-e152-433a-ac16-47aa20629372 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:12.660241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0a1196f2-e152-433a-ac16-47aa20629372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:12.671812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64256b04-c01e-467d-9874-41d7de359110 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:12.672335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64256b04-c01e-467d-9874-41d7de359110 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:12.672830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64256b04-c01e-467d-9874-41d7de359110 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:12.675646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 169/320] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:12.684238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372f8d5-11e5-452d-86d4-1ad40a2e4c73 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:12.684736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372f8d5-11e5-452d-86d4-1ad40a2e4c73 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:12.685245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372f8d5-11e5-452d-86d4-1ad40a2e4c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:12.712409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7372f8d5-11e5-452d-86d4-1ad40a2e4c73 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:12.713707 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7372f8d5-11e5-452d-86d4-1ad40a2e4c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:12.750313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 152/321] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:12.805627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 170/322] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:08:13.013082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1196f2-e152-433a-ac16-47aa20629372 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:13.086428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372f8d5-11e5-452d-86d4-1ad40a2e4c73 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:13.139036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:13.144666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:13.170966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1196f2-e152-433a-ac16-47aa20629372 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:13.216453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc235b-d206-4cae-9f95-2a7dbaba73ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:13.217118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc235b-d206-4cae-9f95-2a7dbaba73ba None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:13.217395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc235b-d206-4cae-9f95-2a7dbaba73ba None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:13.217662 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc235b-d206-4cae-9f95-2a7dbaba73ba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:13.230135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc235b-d206-4cae-9f95-2a7dbaba73ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-41cc235b-d206-4cae-9f95-2a7dbaba73ba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:13.236152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:13.237122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:13.237912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 153/323] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:13.245341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc235b-d206-4cae-9f95-2a7dbaba73ba None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:13.247745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372f8d5-11e5-452d-86d4-1ad40a2e4c73 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:13.297610 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda3f93-b2cb-4674-a4e3-1603c316182b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:13.298710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda3f93-b2cb-4674-a4e3-1603c316182b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:13.299058 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda3f93-b2cb-4674-a4e3-1603c316182b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:13.299405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda3f93-b2cb-4674-a4e3-1603c316182b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:13.300744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda3f93-b2cb-4674-a4e3-1603c316182b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:13.302584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda3f93-b2cb-4674-a4e3-1603c316182b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:13.304205 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 171/324] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:13.310418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda3f93-b2cb-4674-a4e3-1603c316182b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:13.393993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc235b-d206-4cae-9f95-2a7dbaba73ba None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:13.438275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcaa55-c4c4-4464-a324-9fb27da02099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:13.439245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcaa55-c4c4-4464-a324-9fb27da02099 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:13.439673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcaa55-c4c4-4464-a324-9fb27da02099 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:13.440098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcaa55-c4c4-4464-a324-9fb27da02099 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:13.441079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcaa55-c4c4-4464-a324-9fb27da02099 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:13.441518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcaa55-c4c4-4464-a324-9fb27da02099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:13.443505 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcaa55-c4c4-4464-a324-9fb27da02099 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:13.452193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:13.453750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda3f93-b2cb-4674-a4e3-1603c316182b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:13.463530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 154/325] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:13.464680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcaa55-c4c4-4464-a324-9fb27da02099 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:13.468469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:13.499692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab76850-9f2b-4a06-8521-57b68bf1cd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:13.500569 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab76850-9f2b-4a06-8521-57b68bf1cd27 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:13.500569 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab76850-9f2b-4a06-8521-57b68bf1cd27 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:13.500853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab76850-9f2b-4a06-8521-57b68bf1cd27 None admin] PATH_INFO: `/v3/groups` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:13.501451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab76850-9f2b-4a06-8521-57b68bf1cd27 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:13.502968 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab76850-9f2b-4a06-8521-57b68bf1cd27 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:13.525939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 172/326] 10.222.0.64 () {60 vars in 1134 bytes} [Wed Sep 21 16:08:13 2022] POST /identity/v3/groups => generated 226 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:08:13.644400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcaa55-c4c4-4464-a324-9fb27da02099 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:13.706085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc14394-20f9-4c4e-b60d-f87d27aadad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:13.707116 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc14394-20f9-4c4e-b60d-f87d27aadad9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:13.707540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc14394-20f9-4c4e-b60d-f87d27aadad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:13.707956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc14394-20f9-4c4e-b60d-f87d27aadad9 None admin] PATH_INFO: `/v3/users/f88f34f0da6c4c68b99065891776fa4c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:13.710115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc14394-20f9-4c4e-b60d-f87d27aadad9 None admin] RBAC: Authorizing `identity:get_user(user_id=f88f34f0da6c4c68b99065891776fa4c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:13.712616 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc14394-20f9-4c4e-b60d-f87d27aadad9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:13.722576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 155/327] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:08:13 2022] GET /identity/v3/users/f88f34f0da6c4c68b99065891776fa4c => generated 293 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:13.725436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab76850-9f2b-4a06-8521-57b68bf1cd27 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:13.892065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab76850-9f2b-4a06-8521-57b68bf1cd27 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:13.948472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5dded0-d92c-4430-bcc7-df974e86ba29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:13.949531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dded0-d92c-4430-bcc7-df974e86ba29 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:13.950487 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dded0-d92c-4430-bcc7-df974e86ba29 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:13.953149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5dded0-d92c-4430-bcc7-df974e86ba29 None admin] PATH_INFO: `/v3/system/users/f88f34f0da6c4c68b99065891776fa4c/roles/bbbfb342bd244610906d836359f61fcf` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:13.954960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dded0-d92c-4430-bcc7-df974e86ba29 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f88f34f0da6c4c68b99065891776fa4c, role_id=bbbfb342bd244610906d836359f61fcf)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:13.955646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5dded0-d92c-4430-bcc7-df974e86ba29 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:13.966380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 173/328] 10.222.0.64 () {58 vars in 1334 bytes} [Wed Sep 21 16:08:13 2022] PUT /identity/v3/system/users/f88f34f0da6c4c68b99065891776fa4c/roles/bbbfb342bd244610906d836359f61fcf => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:08:14.351204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcba07-7165-4d66-aef7-2c34961d3403 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:14.351425 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcba07-7165-4d66-aef7-2c34961d3403 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:14.351708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcba07-7165-4d66-aef7-2c34961d3403 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:14.354318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 156/329] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:14.365184 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2817bc-b88d-45cd-9548-9fd7d0e4885b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:14.365552 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2817bc-b88d-45cd-9548-9fd7d0e4885b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:14.365801 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2817bc-b88d-45cd-9548-9fd7d0e4885b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:14.398295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0e2817bc-b88d-45cd-9548-9fd7d0e4885b None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:14.399570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0e2817bc-b88d-45cd-9548-9fd7d0e4885b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:14.452767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:14.531877 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 174/330] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:15.099337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcba07-7165-4d66-aef7-2c34961d3403 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:15.243437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:15.339351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcba07-7165-4d66-aef7-2c34961d3403 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:15.386420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c29e74-82d3-4f72-9f65-5df361487df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:15.386980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c29e74-82d3-4f72-9f65-5df361487df3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:15.387257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c29e74-82d3-4f72-9f65-5df361487df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:15.388092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c29e74-82d3-4f72-9f65-5df361487df3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:15.404453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c29e74-82d3-4f72-9f65-5df361487df3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-19c29e74-82d3-4f72-9f65-5df361487df3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:15.410065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:15.412797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:15.415960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 157/331] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:08:15.418494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2817bc-b88d-45cd-9548-9fd7d0e4885b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:15.681018 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2817bc-b88d-45cd-9548-9fd7d0e4885b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:15.756614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cb350-5823-4ac0-b3af-8395b736f0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:15.757295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cb350-5823-4ac0-b3af-8395b736f0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:15.757686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cb350-5823-4ac0-b3af-8395b736f0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:15.758081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cb350-5823-4ac0-b3af-8395b736f0f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:15.758797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cb350-5823-4ac0-b3af-8395b736f0f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:15.759109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cb350-5823-4ac0-b3af-8395b736f0f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:15.760930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cb350-5823-4ac0-b3af-8395b736f0f9 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:15.774462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 175/332] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:15.781214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c29e74-82d3-4f72-9f65-5df361487df3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:15.915405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c29e74-82d3-4f72-9f65-5df361487df3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:15.948776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:15.957039 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4711b7a3-3d68-403d-8f06-c378ebfd3c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:15.957687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711b7a3-3d68-403d-8f06-c378ebfd3c3e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:15.958059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711b7a3-3d68-403d-8f06-c378ebfd3c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:15.958253 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711b7a3-3d68-403d-8f06-c378ebfd3c3e None admin] PATH_INFO: `/v3/users/918584c1108743d29c1bc83297fb7a5d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:15.963540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711b7a3-3d68-403d-8f06-c378ebfd3c3e None admin] RBAC: Authorizing `identity:get_user(user_id=918584c1108743d29c1bc83297fb7a5d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:15.965004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4711b7a3-3d68-403d-8f06-c378ebfd3c3e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:15.967568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 158/333] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:08:15 2022] GET /identity/v3/users/918584c1108743d29c1bc83297fb7a5d => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:15.973970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cb350-5823-4ac0-b3af-8395b736f0f9 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:16.032504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:16.138826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cb350-5823-4ac0-b3af-8395b736f0f9 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:16.186434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964371d1-7616-446b-8019-04f040c8291e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:16.187056 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964371d1-7616-446b-8019-04f040c8291e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:16.187331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964371d1-7616-446b-8019-04f040c8291e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:16.187601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964371d1-7616-446b-8019-04f040c8291e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:16.188346 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964371d1-7616-446b-8019-04f040c8291e None admin] RBAC: Adding query filter params (role.id=20378c15bf524a4c960b49d2c53cf6b2, user.id=918584c1108743d29c1bc83297fb7a5d, scope.system=all) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:16.188651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964371d1-7616-446b-8019-04f040c8291e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:16.190119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964371d1-7616-446b-8019-04f040c8291e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:16.207214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 176/334] 10.222.0.64 () {56 vars in 1308 bytes} [Wed Sep 21 16:08:15 2022] GET /identity/v3/role_assignments?user.id=918584c1108743d29c1bc83297fb7a5d&scope.system=all&role.id=20378c15bf524a4c960b49d2c53cf6b2 => generated 510 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:17.687748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e3356-b809-480c-b15e-bb8381c3076c None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:17.688274 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e3356-b809-480c-b15e-bb8381c3076c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:17.688600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e3356-b809-480c-b15e-bb8381c3076c None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:17.699283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 159/335] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:17 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:17.706466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0076ff-98d2-4166-8b58-44b22e335076 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:17.707300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0076ff-98d2-4166-8b58-44b22e335076 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:17.707432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0076ff-98d2-4166-8b58-44b22e335076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:17.746648 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5e0076ff-98d2-4166-8b58-44b22e335076 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:17.749186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5e0076ff-98d2-4166-8b58-44b22e335076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:17.888198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 177/336] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:18.101465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c0d74-4c6d-474a-a5bc-ad50d9bc50e6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:18.101914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c0d74-4c6d-474a-a5bc-ad50d9bc50e6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:18.102308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c0d74-4c6d-474a-a5bc-ad50d9bc50e6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:18.104691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 160/337] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:18.114156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a65a3-d18b-4e71-a6ca-d92b62063139 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:18.114616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a65a3-d18b-4e71-a6ca-d92b62063139 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:18.115096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a65a3-d18b-4e71-a6ca-d92b62063139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:18.150787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-339a65a3-d18b-4e71-a6ca-d92b62063139 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:18.153235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-339a65a3-d18b-4e71-a6ca-d92b62063139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:18.179039 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:18.298174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 178/338] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:18.523088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c0d74-4c6d-474a-a5bc-ad50d9bc50e6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:18.545014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f5874-78f0-4d7f-8bfe-ba748c5c5b27 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:18.545918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f5874-78f0-4d7f-8bfe-ba748c5c5b27 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:18.546701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f5874-78f0-4d7f-8bfe-ba748c5c5b27 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:18.549516 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 179/339] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:18 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:08:18.563322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d180cb-f8cc-4d17-b10f-258f146b453c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:18.564223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d180cb-f8cc-4d17-b10f-258f146b453c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:18.564996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d180cb-f8cc-4d17-b10f-258f146b453c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:18.630180 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-58d180cb-f8cc-4d17-b10f-258f146b453c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:18.633555 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-58d180cb-f8cc-4d17-b10f-258f146b453c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:18.749568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 180/340] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:08:18.752227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fab3db-9359-4417-b10a-1d9df0ce517c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:18.752836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fab3db-9359-4417-b10a-1d9df0ce517c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:18.753308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fab3db-9359-4417-b10a-1d9df0ce517c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:18.755713 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 181/341] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:18.764364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335e491-e782-4bde-b457-43abf9c65aee None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:18.765022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335e491-e782-4bde-b457-43abf9c65aee None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:18.765520 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335e491-e782-4bde-b457-43abf9c65aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:18.797127 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5335e491-e782-4bde-b457-43abf9c65aee None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:18.798967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5335e491-e782-4bde-b457-43abf9c65aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:18.816682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c0d74-4c6d-474a-a5bc-ad50d9bc50e6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:18.916417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 182/342] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:18.927103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f072b56-249b-4e32-9313-a9ae22b5a2eb None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:18.927398 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f072b56-249b-4e32-9313-a9ae22b5a2eb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:18.927661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f072b56-249b-4e32-9313-a9ae22b5a2eb None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:18.929156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 183/343] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:08:18.936242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebfcbd-cc36-4685-a2bd-4dc24d2de712 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:18.936535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebfcbd-cc36-4685-a2bd-4dc24d2de712 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:18.936797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebfcbd-cc36-4685-a2bd-4dc24d2de712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:18.941606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15dea5-f20b-4579-baae-181cdbf7c28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:18.942775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15dea5-f20b-4579-baae-181cdbf7c28c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:18.943351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15dea5-f20b-4579-baae-181cdbf7c28c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:18.943819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15dea5-f20b-4579-baae-181cdbf7c28c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:18.954034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:18.958834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15dea5-f20b-4579-baae-181cdbf7c28c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-9a15dea5-f20b-4579-baae-181cdbf7c28c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:18.964112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:18.965035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:18.966414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:18.966414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:18.966414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:18.966414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:18.966414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:18.966414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:18.966414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:18.966414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:18.966414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:18.966414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:18.966414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:18.966414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:18.969124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 161/344] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:08:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:18.973667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-89ebfcbd-cc36-4685-a2bd-4dc24d2de712 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:18.973667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-89ebfcbd-cc36-4685-a2bd-4dc24d2de712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:18.974977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15dea5-f20b-4579-baae-181cdbf7c28c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:19.039725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:19.087784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15dea5-f20b-4579-baae-181cdbf7c28c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:19.109327 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 184/345] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:08:19.114072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebfcbd-cc36-4685-a2bd-4dc24d2de712 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:19.139800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cc99c-6087-407a-be5c-360429c6025a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:19.140805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cc99c-6087-407a-be5c-360429c6025a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:19.141309 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cc99c-6087-407a-be5c-360429c6025a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:19.141795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2cc99c-6087-407a-be5c-360429c6025a None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:19.142724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cc99c-6087-407a-be5c-360429c6025a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:19.143257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cc99c-6087-407a-be5c-360429c6025a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:19.144881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2cc99c-6087-407a-be5c-360429c6025a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:19.158756 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 162/346] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:19.175583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cc99c-6087-407a-be5c-360429c6025a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:19.319021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebfcbd-cc36-4685-a2bd-4dc24d2de712 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:19.332157 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2cc99c-6087-407a-be5c-360429c6025a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:19.366627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:19.384807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221c21a-2d97-4884-95dc-b67a8b01aa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:19.385774 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221c21a-2d97-4884-95dc-b67a8b01aa56 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:19.386302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221c21a-2d97-4884-95dc-b67a8b01aa56 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:19.386775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221c21a-2d97-4884-95dc-b67a8b01aa56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:19.401004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041cca8-cc6b-4230-8203-119e518edeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:19.401669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041cca8-cc6b-4230-8203-119e518edeb1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:19.401950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041cca8-cc6b-4230-8203-119e518edeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:19.406200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9041cca8-cc6b-4230-8203-119e518edeb1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:19.406200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:19.428427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9041cca8-cc6b-4230-8203-119e518edeb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-9041cca8-cc6b-4230-8203-119e518edeb1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:19.438077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:19.439032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:19.439819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 185/347] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:19.443289 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041cca8-cc6b-4230-8203-119e518edeb1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:19.509786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221c21a-2d97-4884-95dc-b67a8b01aa56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:19.511253 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221c21a-2d97-4884-95dc-b67a8b01aa56 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:19.704234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 163/348] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 16:08:19.704515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221c21a-2d97-4884-95dc-b67a8b01aa56 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:19.714097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9041cca8-cc6b-4230-8203-119e518edeb1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:19.741178 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:19.779318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194b053-9c86-43d3-9046-46818ade1eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:19.780098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194b053-9c86-43d3-9046-46818ade1eff None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:19.780222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194b053-9c86-43d3-9046-46818ade1eff None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:19.780558 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194b053-9c86-43d3-9046-46818ade1eff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:19.794962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194b053-9c86-43d3-9046-46818ade1eff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-3194b053-9c86-43d3-9046-46818ade1eff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:19.800152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:19.801920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 186/349] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:08:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:19.803556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194b053-9c86-43d3-9046-46818ade1eff None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:19.899674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:19.918460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221c21a-2d97-4884-95dc-b67a8b01aa56 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:19.979078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70340704-7d19-44cd-87cd-584fcc37b610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:19.979078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70340704-7d19-44cd-87cd-584fcc37b610 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:19.979461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70340704-7d19-44cd-87cd-584fcc37b610 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:19.979855 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70340704-7d19-44cd-87cd-584fcc37b610 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:19.980315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70340704-7d19-44cd-87cd-584fcc37b610 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:19.980622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70340704-7d19-44cd-87cd-584fcc37b610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:19.982804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70340704-7d19-44cd-87cd-584fcc37b610 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:19.993513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 164/350] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:19.994459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70340704-7d19-44cd-87cd-584fcc37b610 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:20.008971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194b053-9c86-43d3-9046-46818ade1eff None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:20.062777 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22955a9-a6a2-40db-bf05-6d02feaa47a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:20.064507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22955a9-a6a2-40db-bf05-6d02feaa47a8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:20.064954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22955a9-a6a2-40db-bf05-6d02feaa47a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:20.065227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22955a9-a6a2-40db-bf05-6d02feaa47a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:20.080713 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22955a9-a6a2-40db-bf05-6d02feaa47a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e22955a9-a6a2-40db-bf05-6d02feaa47a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:20.086131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:20.087059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:20.087868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 187/351] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:20.089318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22955a9-a6a2-40db-bf05-6d02feaa47a8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:20.172416 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:20.208556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70340704-7d19-44cd-87cd-584fcc37b610 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:20.247011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22955a9-a6a2-40db-bf05-6d02feaa47a8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:20.265226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:20.275650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb58d16-b2b8-4a6b-9984-51ce7f01920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:20.277056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb58d16-b2b8-4a6b-9984-51ce7f01920e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:20.278324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb58d16-b2b8-4a6b-9984-51ce7f01920e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:20.279012 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb58d16-b2b8-4a6b-9984-51ce7f01920e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:20.294538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e399a-c05a-4efc-ba1c-965540d483cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:20.295195 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e399a-c05a-4efc-ba1c-965540d483cc None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:20.295469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e399a-c05a-4efc-ba1c-965540d483cc None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:20.295739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e399a-c05a-4efc-ba1c-965540d483cc None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:20.297391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e399a-c05a-4efc-ba1c-965540d483cc None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:20.298040 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb58d16-b2b8-4a6b-9984-51ce7f01920e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:20.299077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e399a-c05a-4efc-ba1c-965540d483cc None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:20.300924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 188/352] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:20 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:20.301547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e399a-c05a-4efc-ba1c-965540d483cc None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-dcb58d16-b2b8-4a6b-9984-51ce7f01920e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:20.309895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:20.310875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:20.311664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:20.311664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:20.311664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:20.311664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:20.311664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:20.311664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:20.311664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:20.311664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:20.311664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:20.311664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:20.311664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:20.311664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:20.312276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 165/353] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:20.313749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb58d16-b2b8-4a6b-9984-51ce7f01920e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:20.438988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e399a-c05a-4efc-ba1c-965540d483cc None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:20.470825 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb58d16-b2b8-4a6b-9984-51ce7f01920e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:20.500474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ddadc-61df-401d-a249-7dfc8bc71b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:20.501684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ddadc-61df-401d-a249-7dfc8bc71b50 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:20.501684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ddadc-61df-401d-a249-7dfc8bc71b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:20.503337 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ddadc-61df-401d-a249-7dfc8bc71b50 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:20.504450 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ddadc-61df-401d-a249-7dfc8bc71b50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:20.505795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ddadc-61df-401d-a249-7dfc8bc71b50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:20.507137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ddadc-61df-401d-a249-7dfc8bc71b50 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:20.520430 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 189/354] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:20.521598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ddadc-61df-401d-a249-7dfc8bc71b50 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:20.531424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19666fb7-2822-48a9-b4c5-c168986a8a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:20.532083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19666fb7-2822-48a9-b4c5-c168986a8a73 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:20.532366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19666fb7-2822-48a9-b4c5-c168986a8a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:20.532757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19666fb7-2822-48a9-b4c5-c168986a8a73 None admin] PATH_INFO: `/v3/users/f88f34f0da6c4c68b99065891776fa4c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:20.534343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19666fb7-2822-48a9-b4c5-c168986a8a73 None admin] RBAC: Authorizing `identity:get_user(user_id=f88f34f0da6c4c68b99065891776fa4c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:20.537653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19666fb7-2822-48a9-b4c5-c168986a8a73 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:20.539937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 166/355] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:08:20 2022] GET /identity/v3/users/f88f34f0da6c4c68b99065891776fa4c => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:20.544575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f50b4-75cb-4aa5-869b-ab2ddfae72ea None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:20.544575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f50b4-75cb-4aa5-869b-ab2ddfae72ea None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:20.544575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f50b4-75cb-4aa5-869b-ab2ddfae72ea None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:20.545679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 167/356] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:20.546819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f50b4-75cb-4aa5-869b-ab2ddfae72ea None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:20.563086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:20.586529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:20.681890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ddadc-61df-401d-a249-7dfc8bc71b50 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:20.704048 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f50b4-75cb-4aa5-869b-ab2ddfae72ea None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:20.735927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298dd96-bde8-46c3-a478-67cfedfc01f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:20.736722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298dd96-bde8-46c3-a478-67cfedfc01f0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:20.737146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298dd96-bde8-46c3-a478-67cfedfc01f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:20.737581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298dd96-bde8-46c3-a478-67cfedfc01f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:20.739686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298dd96-bde8-46c3-a478-67cfedfc01f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:20.739686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298dd96-bde8-46c3-a478-67cfedfc01f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:20.741245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298dd96-bde8-46c3-a478-67cfedfc01f0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:20.751164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b406845-c24d-4bbc-9d99-f5ea39c8142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:20.752342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b406845-c24d-4bbc-9d99-f5ea39c8142f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:20.752650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b406845-c24d-4bbc-9d99-f5ea39c8142f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:20.752891 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b406845-c24d-4bbc-9d99-f5ea39c8142f None admin] PATH_INFO: `/v3/groups/5116786fe3844c22b5af4e080a817a3d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:20.754399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 190/357] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:20.761018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b406845-c24d-4bbc-9d99-f5ea39c8142f None admin] RBAC: Authorizing `identity:get_group(group_id=5116786fe3844c22b5af4e080a817a3d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:20.761392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298dd96-bde8-46c3-a478-67cfedfc01f0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:20.763219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b406845-c24d-4bbc-9d99-f5ea39c8142f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:20.765194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 168/358] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:20 2022] GET /identity/v3/groups/5116786fe3844c22b5af4e080a817a3d => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:20.765788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b406845-c24d-4bbc-9d99-f5ea39c8142f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:20.905384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:20.921347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298dd96-bde8-46c3-a478-67cfedfc01f0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:20.930409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:20.961731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b406845-c24d-4bbc-9d99-f5ea39c8142f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:20.983825 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60349c7d-0734-4c14-a9da-53a5c17eeae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:20.984640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60349c7d-0734-4c14-a9da-53a5c17eeae6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:20.989414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60349c7d-0734-4c14-a9da-53a5c17eeae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:20.989414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60349c7d-0734-4c14-a9da-53a5c17eeae6 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:20.989414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60349c7d-0734-4c14-a9da-53a5c17eeae6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:20.989414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60349c7d-0734-4c14-a9da-53a5c17eeae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:20.990339 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60349c7d-0734-4c14-a9da-53a5c17eeae6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:21.004795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 191/359] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:21.004795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60349c7d-0734-4c14-a9da-53a5c17eeae6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:21.026348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f53a23-b481-412b-9775-c9f01f62704f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:21.027160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f53a23-b481-412b-9775-c9f01f62704f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:21.028150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f53a23-b481-412b-9775-c9f01f62704f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:21.028617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f53a23-b481-412b-9775-c9f01f62704f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:21.163688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f53a23-b481-412b-9775-c9f01f62704f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:21.166188 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f53a23-b481-412b-9775-c9f01f62704f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:21.180157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60349c7d-0734-4c14-a9da-53a5c17eeae6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:21.234849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97005776-c569-4ea1-8252-9ab19d7905ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:21.235632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005776-c569-4ea1-8252-9ab19d7905ab None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:21.236900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005776-c569-4ea1-8252-9ab19d7905ab None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:21.237267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97005776-c569-4ea1-8252-9ab19d7905ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:21.238174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97005776-c569-4ea1-8252-9ab19d7905ab None admin] RBAC: Adding query filter params (role.id=bbbfb342bd244610906d836359f61fcf, user.id=f88f34f0da6c4c68b99065891776fa4c, scope.system=all) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:21.238694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97005776-c569-4ea1-8252-9ab19d7905ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:21.241414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97005776-c569-4ea1-8252-9ab19d7905ab None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:21.262279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 192/360] 10.222.0.64 () {56 vars in 1308 bytes} [Wed Sep 21 16:08:21 2022] GET /identity/v3/role_assignments?user.id=f88f34f0da6c4c68b99065891776fa4c&scope.system=all&role.id=bbbfb342bd244610906d836359f61fcf => generated 510 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:21.264236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165efab1-7386-413e-aad7-6d1078f7ab12 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:21.264690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165efab1-7386-413e-aad7-6d1078f7ab12 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:21.265139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165efab1-7386-413e-aad7-6d1078f7ab12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:21.296998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-165efab1-7386-413e-aad7-6d1078f7ab12 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:21.297443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:21.300055 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-165efab1-7386-413e-aad7-6d1078f7ab12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:21.349366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:21.365529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 169/361] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:21.367639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f53a23-b481-412b-9775-c9f01f62704f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:21.444498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 193/362] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:21.445985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165efab1-7386-413e-aad7-6d1078f7ab12 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:21.601846 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f53a23-b481-412b-9775-c9f01f62704f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:21.670936 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165efab1-7386-413e-aad7-6d1078f7ab12 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:21.681827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddea08-9e9d-433b-9896-304800cf874a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:21.684017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddea08-9e9d-433b-9896-304800cf874a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:21.684889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddea08-9e9d-433b-9896-304800cf874a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:21.685565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddea08-9e9d-433b-9896-304800cf874a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:21.767452 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b3f85-87af-466d-82f9-d659aba8daf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:21.768568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b3f85-87af-466d-82f9-d659aba8daf1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:21.769123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b3f85-87af-466d-82f9-d659aba8daf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:21.770103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:21.771207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b3f85-87af-466d-82f9-d659aba8daf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:21.772836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b3f85-87af-466d-82f9-d659aba8daf1 None admin] RBAC: Adding query filter params (group.id=5116786fe3844c22b5af4e080a817a3d, role.id=aa320f864d914f65bebe5463994ff536, scope.project.id=832b79a184af40d7ad323b5e97816800) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:21.773620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b3f85-87af-466d-82f9-d659aba8daf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:21.776446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b3f85-87af-466d-82f9-d659aba8daf1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:21.788286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 194/363] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Sep 21 16:08:21 2022] GET /identity/v3/role_assignments?group.id=5116786fe3844c22b5af4e080a817a3d&scope.project.id=832b79a184af40d7ad323b5e97816800&role.id=aa320f864d914f65bebe5463994ff536 => generated 260 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:21.789999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b3f85-87af-466d-82f9-d659aba8daf1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:21.823772 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:21.838471 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddea08-9e9d-433b-9896-304800cf874a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:21.841583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddea08-9e9d-433b-9896-304800cf874a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:21.928556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b3f85-87af-466d-82f9-d659aba8daf1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:21.973378 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a6132-38da-4992-aeb6-f3e8363e097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:21.974007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a6132-38da-4992-aeb6-f3e8363e097e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:21.974411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a6132-38da-4992-aeb6-f3e8363e097e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:21.974788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a6132-38da-4992-aeb6-f3e8363e097e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:22.042121 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 170/364] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:22.044492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddea08-9e9d-433b-9896-304800cf874a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:22.074359 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a6132-38da-4992-aeb6-f3e8363e097e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:22.076852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437a6132-38da-4992-aeb6-f3e8363e097e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:22.111887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:22.171239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:22.211895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddea08-9e9d-433b-9896-304800cf874a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:22.254556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f4327-cb83-4f18-8114-59c544835e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:22.255212 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f4327-cb83-4f18-8114-59c544835e5e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:22.255500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f4327-cb83-4f18-8114-59c544835e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:22.255764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f4327-cb83-4f18-8114-59c544835e5e None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:22.256943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f4327-cb83-4f18-8114-59c544835e5e None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:22.259158 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f4327-cb83-4f18-8114-59c544835e5e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:22.261211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 171/365] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:22 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:22.262241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f4327-cb83-4f18-8114-59c544835e5e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:22.266107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 195/366] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:22.266107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a6132-38da-4992-aeb6-f3e8363e097e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:22.398988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f4327-cb83-4f18-8114-59c544835e5e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:22.402121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a6132-38da-4992-aeb6-f3e8363e097e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:22.439421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:22.459520 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06e9f2-4785-4207-be83-d09f8df57cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:22.460549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06e9f2-4785-4207-be83-d09f8df57cce None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:22.460986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06e9f2-4785-4207-be83-d09f8df57cce None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:22.461444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06e9f2-4785-4207-be83-d09f8df57cce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:22.463736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203aad66-d5ed-4802-8a50-e6487b1d906e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:22.464587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203aad66-d5ed-4802-8a50-e6487b1d906e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:22.465586 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203aad66-d5ed-4802-8a50-e6487b1d906e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:22.466069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203aad66-d5ed-4802-8a50-e6487b1d906e None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:22.467908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203aad66-d5ed-4802-8a50-e6487b1d906e None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:22.470480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203aad66-d5ed-4802-8a50-e6487b1d906e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:22.473403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 196/367] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:22 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:22.474410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203aad66-d5ed-4802-8a50-e6487b1d906e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:22.488582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:22.490303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06e9f2-4785-4207-be83-d09f8df57cce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-3a06e9f2-4785-4207-be83-d09f8df57cce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:22.501033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:22.502112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:22.502914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 172/368] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:08:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:22.504322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06e9f2-4785-4207-be83-d09f8df57cce None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:22.663597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203aad66-d5ed-4802-8a50-e6487b1d906e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:22.692004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06e9f2-4785-4207-be83-d09f8df57cce None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:22.722476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102bfe8-6f59-43fc-89f9-c5268b41de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:22.723386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102bfe8-6f59-43fc-89f9-c5268b41de53 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:22.723855 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102bfe8-6f59-43fc-89f9-c5268b41de53 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:22.724331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102bfe8-6f59-43fc-89f9-c5268b41de53 None admin] PATH_INFO: `/v3/groups/5116786fe3844c22b5af4e080a817a3d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:22.726233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102bfe8-6f59-43fc-89f9-c5268b41de53 None admin] RBAC: Authorizing `identity:get_group(group_id=5116786fe3844c22b5af4e080a817a3d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:22.728799 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102bfe8-6f59-43fc-89f9-c5268b41de53 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:22.732188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 197/369] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:22 2022] GET /identity/v3/groups/5116786fe3844c22b5af4e080a817a3d => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:22.733535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102bfe8-6f59-43fc-89f9-c5268b41de53 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:22.754983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7086c-6af0-41d2-aa1a-b4d1e0fcbe2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:22.756180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7086c-6af0-41d2-aa1a-b4d1e0fcbe2a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:22.756730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7086c-6af0-41d2-aa1a-b4d1e0fcbe2a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:22.757028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd7086c-6af0-41d2-aa1a-b4d1e0fcbe2a None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:22.758293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7086c-6af0-41d2-aa1a-b4d1e0fcbe2a None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:22.760513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd7086c-6af0-41d2-aa1a-b4d1e0fcbe2a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:22.763174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 173/370] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:22 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:22.763709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7086c-6af0-41d2-aa1a-b4d1e0fcbe2a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:22.823689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:22.852649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:22.890457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102bfe8-6f59-43fc-89f9-c5268b41de53 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:22.897789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd7086c-6af0-41d2-aa1a-b4d1e0fcbe2a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:22.936763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36431b7b-c13d-48c0-b104-51ab529618f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:22.937781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36431b7b-c13d-48c0-b104-51ab529618f5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:22.938207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36431b7b-c13d-48c0-b104-51ab529618f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:22.938533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36431b7b-c13d-48c0-b104-51ab529618f5 None admin] PATH_INFO: `/v3/groups/5116786fe3844c22b5af4e080a817a3d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:22.939962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36431b7b-c13d-48c0-b104-51ab529618f5 None admin] RBAC: Authorizing `identity:get_group(group_id=5116786fe3844c22b5af4e080a817a3d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:22.941755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36431b7b-c13d-48c0-b104-51ab529618f5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:22.942997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41975455-ab88-4122-87f6-0ccc17d689ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:22.943693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 198/371] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:22 2022] GET /identity/v3/groups/5116786fe3844c22b5af4e080a817a3d => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:22.944056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41975455-ab88-4122-87f6-0ccc17d689ed None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:22.944537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41975455-ab88-4122-87f6-0ccc17d689ed None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:22.944971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41975455-ab88-4122-87f6-0ccc17d689ed None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:22.945097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36431b7b-c13d-48c0-b104-51ab529618f5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:22.945987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41975455-ab88-4122-87f6-0ccc17d689ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:22.946406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41975455-ab88-4122-87f6-0ccc17d689ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:22.947814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41975455-ab88-4122-87f6-0ccc17d689ed None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:22.959238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 174/372] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:22.960159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41975455-ab88-4122-87f6-0ccc17d689ed None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:23.075276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36431b7b-c13d-48c0-b104-51ab529618f5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:23.095924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41975455-ab88-4122-87f6-0ccc17d689ed None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:23.103911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:23.118776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312b9eb-228c-4631-957f-a915179adc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:23.119781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312b9eb-228c-4631-957f-a915179adc8e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:23.120248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312b9eb-228c-4631-957f-a915179adc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:23.120715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312b9eb-228c-4631-957f-a915179adc8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:23.122640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312b9eb-228c-4631-957f-a915179adc8e None admin] RBAC: Adding query filter params (group.id=5116786fe3844c22b5af4e080a817a3d, role.id=aa320f864d914f65bebe5463994ff536, scope.project.id=6753f2523be140a3a48348df431cf0d9) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:23.123160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312b9eb-228c-4631-957f-a915179adc8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:23.125612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312b9eb-228c-4631-957f-a915179adc8e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:23.134608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:23.134952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 199/373] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Sep 21 16:08:22 2022] GET /identity/v3/role_assignments?group.id=5116786fe3844c22b5af4e080a817a3d&scope.project.id=6753f2523be140a3a48348df431cf0d9&role.id=aa320f864d914f65bebe5463994ff536 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:23.136533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312b9eb-228c-4631-957f-a915179adc8e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:23.155712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1327b0b-8e56-48eb-85e4-6fe0d8603c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:23.157475 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1327b0b-8e56-48eb-85e4-6fe0d8603c0e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:23.158012 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1327b0b-8e56-48eb-85e4-6fe0d8603c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:23.158495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1327b0b-8e56-48eb-85e4-6fe0d8603c0e None admin] PATH_INFO: `/v3/groups/5116786fe3844c22b5af4e080a817a3d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:23.161062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1327b0b-8e56-48eb-85e4-6fe0d8603c0e None admin] RBAC: Authorizing `identity:get_group(group_id=5116786fe3844c22b5af4e080a817a3d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:23.163013 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1327b0b-8e56-48eb-85e4-6fe0d8603c0e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:23.166287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 175/374] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:22 2022] GET /identity/v3/groups/5116786fe3844c22b5af4e080a817a3d => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:23.167888 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1327b0b-8e56-48eb-85e4-6fe0d8603c0e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:23.295646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312b9eb-228c-4631-957f-a915179adc8e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:23.315144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1327b0b-8e56-48eb-85e4-6fe0d8603c0e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:23.338386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeae2a1-0b09-46d0-a07f-f6bb904ce33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:23.339058 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeae2a1-0b09-46d0-a07f-f6bb904ce33e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:23.339414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeae2a1-0b09-46d0-a07f-f6bb904ce33e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:23.339778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeae2a1-0b09-46d0-a07f-f6bb904ce33e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:23.341200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeae2a1-0b09-46d0-a07f-f6bb904ce33e None admin] RBAC: Adding query filter params (group.id=5116786fe3844c22b5af4e080a817a3d, role.id=20378c15bf524a4c960b49d2c53cf6b2, scope.project.id=832b79a184af40d7ad323b5e97816800) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:23.341589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeae2a1-0b09-46d0-a07f-f6bb904ce33e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:23.343236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aeae2a1-0b09-46d0-a07f-f6bb904ce33e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:23.354550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 200/375] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Sep 21 16:08:23 2022] GET /identity/v3/role_assignments?group.id=5116786fe3844c22b5af4e080a817a3d&scope.project.id=832b79a184af40d7ad323b5e97816800&role.id=20378c15bf524a4c960b49d2c53cf6b2 => generated 260 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:23.355706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeae2a1-0b09-46d0-a07f-f6bb904ce33e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:23.363400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9becb71-6f6b-48be-a0ef-429369ba3a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:23.364062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9becb71-6f6b-48be-a0ef-429369ba3a68 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:23.364319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9becb71-6f6b-48be-a0ef-429369ba3a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:23.364592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9becb71-6f6b-48be-a0ef-429369ba3a68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:23.475018 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:23.521072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:23.529479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9becb71-6f6b-48be-a0ef-429369ba3a68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:23.531064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9becb71-6f6b-48be-a0ef-429369ba3a68 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:23.577091 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeae2a1-0b09-46d0-a07f-f6bb904ce33e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:23.642658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd0170-64e4-472e-a8a7-2ecad61185ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:23.643208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd0170-64e4-472e-a8a7-2ecad61185ce None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:23.643491 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd0170-64e4-472e-a8a7-2ecad61185ce None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:23.643762 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd0170-64e4-472e-a8a7-2ecad61185ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:23.644514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd0170-64e4-472e-a8a7-2ecad61185ce None admin] RBAC: Adding query filter params (group.id=5116786fe3844c22b5af4e080a817a3d, role.id=20378c15bf524a4c960b49d2c53cf6b2, scope.project.id=6753f2523be140a3a48348df431cf0d9) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:23.644827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd0170-64e4-472e-a8a7-2ecad61185ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:23.646311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd0170-64e4-472e-a8a7-2ecad61185ce None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:23.664009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 201/376] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Sep 21 16:08:23 2022] GET /identity/v3/role_assignments?group.id=5116786fe3844c22b5af4e080a817a3d&scope.project.id=6753f2523be140a3a48348df431cf0d9&role.id=20378c15bf524a4c960b49d2c53cf6b2 => generated 260 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:23.758261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 176/377] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:23.767379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd0170-64e4-472e-a8a7-2ecad61185ce None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:24.009340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd0170-64e4-472e-a8a7-2ecad61185ce None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:24.051242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:24.093639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5944208-ae71-4982-b64d-c49d2f7dfa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:24.094828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5944208-ae71-4982-b64d-c49d2f7dfa63 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:24.095334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5944208-ae71-4982-b64d-c49d2f7dfa63 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:24.095825 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5944208-ae71-4982-b64d-c49d2f7dfa63 None admin] PATH_INFO: `/v3/projects/92877a675fac490b8473d8157aef4217` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:24.098954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5944208-ae71-4982-b64d-c49d2f7dfa63 None admin] RBAC: Authorizing `identity:get_project(project_id=92877a675fac490b8473d8157aef4217)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:24.100510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5944208-ae71-4982-b64d-c49d2f7dfa63 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:24.102351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 202/378] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:23 2022] GET /identity/v3/projects/92877a675fac490b8473d8157aef4217 => generated 344 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:24.109534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9becb71-6f6b-48be-a0ef-429369ba3a68 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:24.316958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:24.321821 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9becb71-6f6b-48be-a0ef-429369ba3a68 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:24.375470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72218e19-e620-42f8-815b-70ce2a348060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:24.376130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72218e19-e620-42f8-815b-70ce2a348060 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:24.376404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72218e19-e620-42f8-815b-70ce2a348060 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:24.376678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72218e19-e620-42f8-815b-70ce2a348060 None admin] PATH_INFO: `/v3/groups/d6b80730bdbf48fb93a94fb0197bb9bb` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:24.385989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72218e19-e620-42f8-815b-70ce2a348060 None admin] RBAC: Authorizing `identity:get_group(group_id=d6b80730bdbf48fb93a94fb0197bb9bb)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:24.388047 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72218e19-e620-42f8-815b-70ce2a348060 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:24.390102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 177/379] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:24 2022] GET /identity/v3/groups/d6b80730bdbf48fb93a94fb0197bb9bb => generated 229 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:24.395814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5944208-ae71-4982-b64d-c49d2f7dfa63 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:24.581343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5944208-ae71-4982-b64d-c49d2f7dfa63 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:24.632566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca7ec6-dc7e-4f64-b76b-400651610923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:24.633217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca7ec6-dc7e-4f64-b76b-400651610923 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:24.633502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca7ec6-dc7e-4f64-b76b-400651610923 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:24.633781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca7ec6-dc7e-4f64-b76b-400651610923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:24.634581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca7ec6-dc7e-4f64-b76b-400651610923 None admin] RBAC: Adding query filter params (group.id=d6b80730bdbf48fb93a94fb0197bb9bb, role.id=459d01a218314543a7a8329e88b78331, scope.project.id=92877a675fac490b8473d8157aef4217) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:24.634898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca7ec6-dc7e-4f64-b76b-400651610923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:24.636369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca7ec6-dc7e-4f64-b76b-400651610923 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:24.643383 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 203/380] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Sep 21 16:08:24 2022] GET /identity/v3/role_assignments?group.id=d6b80730bdbf48fb93a94fb0197bb9bb&scope.project.id=92877a675fac490b8473d8157aef4217&role.id=459d01a218314543a7a8329e88b78331 => generated 260 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:25.852904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeca2ed-543e-4daf-82c8-d42e1832bcf3 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:25.854786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeca2ed-543e-4daf-82c8-d42e1832bcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:25.856117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeca2ed-543e-4daf-82c8-d42e1832bcf3 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:25.859101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 178/381] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:25.866403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494f393-09c8-4330-9a28-e55c333bfabb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:25.867982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494f393-09c8-4330-9a28-e55c333bfabb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:25.868768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1494f393-09c8-4330-9a28-e55c333bfabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:25.902843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1494f393-09c8-4330-9a28-e55c333bfabb None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:25.905447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1494f393-09c8-4330-9a28-e55c333bfabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:26.029031 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:26.087086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 204/382] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:08:26.145436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96887616-8a9e-49fc-82dd-e63449d96139 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:26.146393 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96887616-8a9e-49fc-82dd-e63449d96139 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:26.147151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96887616-8a9e-49fc-82dd-e63449d96139 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:26.150007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 179/383] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:26.159073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f2db1-7768-4580-9bca-6f4a8ebfbb54 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:26.160018 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f2db1-7768-4580-9bca-6f4a8ebfbb54 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:26.160765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f2db1-7768-4580-9bca-6f4a8ebfbb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:26.204620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0e5f2db1-7768-4580-9bca-6f4a8ebfbb54 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:26.207157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0e5f2db1-7768-4580-9bca-6f4a8ebfbb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:26.387652 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 205/384] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:26.569809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96887616-8a9e-49fc-82dd-e63449d96139 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:26.832741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96887616-8a9e-49fc-82dd-e63449d96139 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:26.924379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f2db1-7768-4580-9bca-6f4a8ebfbb54 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:26.928768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4caa0-5105-423f-af00-1378a38635df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:26.931067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4caa0-5105-423f-af00-1378a38635df None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:26.931912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4caa0-5105-423f-af00-1378a38635df None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:26.932678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4caa0-5105-423f-af00-1378a38635df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:26.966390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e4caa0-5105-423f-af00-1378a38635df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-58e4caa0-5105-423f-af00-1378a38635df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:26.974095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:26.975868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:26.977412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 180/385] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:08:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:08:26.979845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4caa0-5105-423f-af00-1378a38635df None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:27.032593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:27.207497 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f2db1-7768-4580-9bca-6f4a8ebfbb54 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:27.233602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e4caa0-5105-423f-af00-1378a38635df None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:27.240132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:27.306802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e2bb5-9f24-4b4c-9a3e-c9750d1d9347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:27.307876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e2bb5-9f24-4b4c-9a3e-c9750d1d9347 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:27.308527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e2bb5-9f24-4b4c-9a3e-c9750d1d9347 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:27.309008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e2bb5-9f24-4b4c-9a3e-c9750d1d9347 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:27.329388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7e2bb5-9f24-4b4c-9a3e-c9750d1d9347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4b7e2bb5-9f24-4b4c-9a3e-c9750d1d9347 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:27.338154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:27.339120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:27.340483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:27.340483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:27.340483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:27.340483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:27.340483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:27.340483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:27.340483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:27.340483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:27.340483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:27.340483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:27.340483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:27.340483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:27.341098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 206/386] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:08:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:27.356780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e2bb5-9f24-4b4c-9a3e-c9750d1d9347 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:27.381274 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064b156-8d35-486c-b5e1-368b1741daeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:27.382329 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064b156-8d35-486c-b5e1-368b1741daeb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:27.382832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064b156-8d35-486c-b5e1-368b1741daeb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:27.383311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064b156-8d35-486c-b5e1-368b1741daeb None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:27.384546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064b156-8d35-486c-b5e1-368b1741daeb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:27.385096 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064b156-8d35-486c-b5e1-368b1741daeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:27.388334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064b156-8d35-486c-b5e1-368b1741daeb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:27.414078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 181/387] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:27.416406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97e824-8073-4a26-a6a4-1de485f9f3a8 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:27.416921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97e824-8073-4a26-a6a4-1de485f9f3a8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:27.417435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97e824-8073-4a26-a6a4-1de485f9f3a8 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:27.420474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 182/388] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:27.427382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97e824-8073-4a26-a6a4-1de485f9f3a8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:27.568580 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e2bb5-9f24-4b4c-9a3e-c9750d1d9347 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:27.583975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:27.618110 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97e824-8073-4a26-a6a4-1de485f9f3a8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:27.624980 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a15519-aaee-4599-8c45-aa0bc2cabf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:27.625749 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a15519-aaee-4599-8c45-aa0bc2cabf69 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:27.626210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a15519-aaee-4599-8c45-aa0bc2cabf69 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:27.626609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a15519-aaee-4599-8c45-aa0bc2cabf69 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:27.627421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a15519-aaee-4599-8c45-aa0bc2cabf69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:27.627846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a15519-aaee-4599-8c45-aa0bc2cabf69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:27.629535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a15519-aaee-4599-8c45-aa0bc2cabf69 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:27.646243 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 207/389] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:27.646243 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07969895-2994-48c2-bf6b-3414da8a2811 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:27.646243 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07969895-2994-48c2-bf6b-3414da8a2811 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:27.646243 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07969895-2994-48c2-bf6b-3414da8a2811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:27.668158 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-07969895-2994-48c2-bf6b-3414da8a2811 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:27.669397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-07969895-2994-48c2-bf6b-3414da8a2811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:27.683807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793c492-02f3-4a9b-ba66-81c9bfb99398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:27.684794 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793c492-02f3-4a9b-ba66-81c9bfb99398 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:27.685288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793c492-02f3-4a9b-ba66-81c9bfb99398 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:27.685758 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793c492-02f3-4a9b-ba66-81c9bfb99398 None admin] PATH_INFO: `/v3/groups/5116786fe3844c22b5af4e080a817a3d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:27.688224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793c492-02f3-4a9b-ba66-81c9bfb99398 None admin] RBAC: Authorizing `identity:get_group(group_id=5116786fe3844c22b5af4e080a817a3d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:27.691026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793c492-02f3-4a9b-ba66-81c9bfb99398 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:27.694141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 183/390] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:27 2022] GET /identity/v3/groups/5116786fe3844c22b5af4e080a817a3d => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:27.696194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f851f5e-7bd2-4d0d-a76b-c808065592e1 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:27.696703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f851f5e-7bd2-4d0d-a76b-c808065592e1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:27.697173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f851f5e-7bd2-4d0d-a76b-c808065592e1 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:27.702292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 184/391] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:27.702292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f851f5e-7bd2-4d0d-a76b-c808065592e1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:27.703923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:27.757278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 208/392] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:27.757889 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07969895-2994-48c2-bf6b-3414da8a2811 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:27.875231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07969895-2994-48c2-bf6b-3414da8a2811 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:27.887564 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f851f5e-7bd2-4d0d-a76b-c808065592e1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:27.919812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafce5e-534d-489a-a2c8-faed61764316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:27.920882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafce5e-534d-489a-a2c8-faed61764316 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:27.921954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafce5e-534d-489a-a2c8-faed61764316 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:27.921954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafce5e-534d-489a-a2c8-faed61764316 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:27.923134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafce5e-534d-489a-a2c8-faed61764316 None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:27.924816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafce5e-534d-489a-a2c8-faed61764316 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:27.932297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 209/393] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:27 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:27.933729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066a8c6-f00a-4422-8bfa-daae2bec9e3c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:27.933990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066a8c6-f00a-4422-8bfa-daae2bec9e3c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:27.934314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066a8c6-f00a-4422-8bfa-daae2bec9e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:27.943758 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97c581-28c1-45ac-aa5c-8cbc7d1eb75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:27.945154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97c581-28c1-45ac-aa5c-8cbc7d1eb75e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:27.945665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97c581-28c1-45ac-aa5c-8cbc7d1eb75e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:27.946170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef97c581-28c1-45ac-aa5c-8cbc7d1eb75e None admin] PATH_INFO: `/v3/groups/5116786fe3844c22b5af4e080a817a3d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:27.947699 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97c581-28c1-45ac-aa5c-8cbc7d1eb75e None admin] RBAC: Authorizing `identity:get_group(group_id=5116786fe3844c22b5af4e080a817a3d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:27.950291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef97c581-28c1-45ac-aa5c-8cbc7d1eb75e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:27.952833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 185/394] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:27 2022] GET /identity/v3/groups/5116786fe3844c22b5af4e080a817a3d => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:27.953836 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97c581-28c1-45ac-aa5c-8cbc7d1eb75e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:27.955564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a066a8c6-f00a-4422-8bfa-daae2bec9e3c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:27.956824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a066a8c6-f00a-4422-8bfa-daae2bec9e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:27.983490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:27.986988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:28.050867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 210/395] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:28.052057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066a8c6-f00a-4422-8bfa-daae2bec9e3c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:28.108931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef97c581-28c1-45ac-aa5c-8cbc7d1eb75e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:28.172024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4827e02-c6df-4776-9094-342ea8d575d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:28.172977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4827e02-c6df-4776-9094-342ea8d575d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:28.173277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4827e02-c6df-4776-9094-342ea8d575d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:28.173596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4827e02-c6df-4776-9094-342ea8d575d0 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800/groups/5116786fe3844c22b5af4e080a817a3d/roles/aa320f864d914f65bebe5463994ff536` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:28.176623 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4827e02-c6df-4776-9094-342ea8d575d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=832b79a184af40d7ad323b5e97816800, group_id=5116786fe3844c22b5af4e080a817a3d, role_id=aa320f864d914f65bebe5463994ff536)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:28.178116 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4827e02-c6df-4776-9094-342ea8d575d0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:28.193094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066a8c6-f00a-4422-8bfa-daae2bec9e3c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:28.197951 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 186/396] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Sep 21 16:08:27 2022] PUT /identity/v3/projects/832b79a184af40d7ad323b5e97816800/groups/5116786fe3844c22b5af4e080a817a3d/roles/aa320f864d914f65bebe5463994ff536 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:08:28.205026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28a347-5856-41c4-bbc7-02a9a6421c8c None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:28.205323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28a347-5856-41c4-bbc7-02a9a6421c8c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:28.205569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc28a347-5856-41c4-bbc7-02a9a6421c8c None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:28.207942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 187/397] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:28.215508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9e4a-a755-424b-8b3a-4034fbe7089d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:28.215885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9e4a-a755-424b-8b3a-4034fbe7089d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:28.216141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b9e4a-a755-424b-8b3a-4034fbe7089d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:28.244401 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-183b9e4a-a755-424b-8b3a-4034fbe7089d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:28.248145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-183b9e4a-a755-424b-8b3a-4034fbe7089d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:28.265390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332860d0-fde5-4cd7-92b0-1810a4e70863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:28.266174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332860d0-fde5-4cd7-92b0-1810a4e70863 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:28.266456 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332860d0-fde5-4cd7-92b0-1810a4e70863 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:28.266729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332860d0-fde5-4cd7-92b0-1810a4e70863 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:28.268015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332860d0-fde5-4cd7-92b0-1810a4e70863 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:28.269505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332860d0-fde5-4cd7-92b0-1810a4e70863 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:28.271598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 211/398] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:28 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:28.282102 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332860d0-fde5-4cd7-92b0-1810a4e70863 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:28.313699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:28.414823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 188/399] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:28.416483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b9e4a-a755-424b-8b3a-4034fbe7089d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:28.421248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332860d0-fde5-4cd7-92b0-1810a4e70863 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:28.442283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:28.502084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1169e4-22d2-4ae6-96c2-c9577b78d77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:28.502084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1169e4-22d2-4ae6-96c2-c9577b78d77f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:28.502502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1169e4-22d2-4ae6-96c2-c9577b78d77f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:28.505420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1169e4-22d2-4ae6-96c2-c9577b78d77f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:28.532333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1169e4-22d2-4ae6-96c2-c9577b78d77f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4c1169e4-22d2-4ae6-96c2-c9577b78d77f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:28.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:28.548984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:28.550946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 212/400] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:28.561497 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1169e4-22d2-4ae6-96c2-c9577b78d77f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:28.579674 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b9e4a-a755-424b-8b3a-4034fbe7089d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:28.624974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fde25-f62b-42a3-9bcb-bcc7b782c7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:28.625687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fde25-f62b-42a3-9bcb-bcc7b782c7ca None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:28.625968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fde25-f62b-42a3-9bcb-bcc7b782c7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:28.626297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fde25-f62b-42a3-9bcb-bcc7b782c7ca None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9/groups/5116786fe3844c22b5af4e080a817a3d/roles/aa320f864d914f65bebe5463994ff536` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:28.628713 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fde25-f62b-42a3-9bcb-bcc7b782c7ca None admin] RBAC: Authorizing `identity:create_grant(project_id=6753f2523be140a3a48348df431cf0d9, group_id=5116786fe3844c22b5af4e080a817a3d, role_id=aa320f864d914f65bebe5463994ff536)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:28.630893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fde25-f62b-42a3-9bcb-bcc7b782c7ca None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:28.646441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 189/401] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Sep 21 16:08:28 2022] PUT /identity/v3/projects/6753f2523be140a3a48348df431cf0d9/groups/5116786fe3844c22b5af4e080a817a3d/roles/aa320f864d914f65bebe5463994ff536 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:08:28.647343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fde25-f62b-42a3-9bcb-bcc7b782c7ca None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:28.750674 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:28.777983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:28.780864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fde25-f62b-42a3-9bcb-bcc7b782c7ca None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:28.830833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1169e4-22d2-4ae6-96c2-c9577b78d77f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:28.861561 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32d19b-975f-423b-9d29-a42f4e805c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:28.862261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32d19b-975f-423b-9d29-a42f4e805c80 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:28.862538 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32d19b-975f-423b-9d29-a42f4e805c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:28.862801 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32d19b-975f-423b-9d29-a42f4e805c80 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:28.883757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb35b6-3e03-4ce2-8027-b30b8a87bad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:28.884381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb35b6-3e03-4ce2-8027-b30b8a87bad7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:28.884659 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb35b6-3e03-4ce2-8027-b30b8a87bad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:28.884909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb35b6-3e03-4ce2-8027-b30b8a87bad7 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:28.885600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb35b6-3e03-4ce2-8027-b30b8a87bad7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:28.885905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb35b6-3e03-4ce2-8027-b30b8a87bad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:28.887915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb35b6-3e03-4ce2-8027-b30b8a87bad7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:28.891143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32d19b-975f-423b-9d29-a42f4e805c80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:28.900300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 213/402] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:28.901187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb35b6-3e03-4ce2-8027-b30b8a87bad7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-3c32d19b-975f-423b-9d29-a42f4e805c80 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:28.905839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:28.907533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:28.909101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:28.909101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:28.909101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:28.909101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:28.909101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:28.909101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:28.909101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:28.909101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:28.909101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:28.909101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:28.909101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:28.909101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:28.911606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 190/403] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:28.918150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32d19b-975f-423b-9d29-a42f4e805c80 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:29.092910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32d19b-975f-423b-9d29-a42f4e805c80 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:29.116992 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb35b6-3e03-4ce2-8027-b30b8a87bad7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:29.161325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:29.169506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27149b24-2804-4e8a-aeba-77e9d5c5fc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:29.170923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27149b24-2804-4e8a-aeba-77e9d5c5fc19 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:29.171490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27149b24-2804-4e8a-aeba-77e9d5c5fc19 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:29.172196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27149b24-2804-4e8a-aeba-77e9d5c5fc19 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:29.174297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de8186-c2b5-46ac-85ed-0fd66753807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:29.176007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de8186-c2b5-46ac-85ed-0fd66753807f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:29.176775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de8186-c2b5-46ac-85ed-0fd66753807f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:29.178017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de8186-c2b5-46ac-85ed-0fd66753807f None admin] PATH_INFO: `/v3/groups/5116786fe3844c22b5af4e080a817a3d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:29.180434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de8186-c2b5-46ac-85ed-0fd66753807f None admin] RBAC: Authorizing `identity:get_group(group_id=5116786fe3844c22b5af4e080a817a3d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:29.184037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de8186-c2b5-46ac-85ed-0fd66753807f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:29.194577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 191/404] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:28 2022] GET /identity/v3/groups/5116786fe3844c22b5af4e080a817a3d => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:29.195792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de8186-c2b5-46ac-85ed-0fd66753807f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:29.198405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27149b24-2804-4e8a-aeba-77e9d5c5fc19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-27149b24-2804-4e8a-aeba-77e9d5c5fc19 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:29.211584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:29.213347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:29.215386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 214/405] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:08:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:29.219291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27149b24-2804-4e8a-aeba-77e9d5c5fc19 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:29.250463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:29.425352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de8186-c2b5-46ac-85ed-0fd66753807f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:29.444154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27149b24-2804-4e8a-aeba-77e9d5c5fc19 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:29.501980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971121bd-479f-4e6f-8848-1a19baa0e15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:29.503595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971121bd-479f-4e6f-8848-1a19baa0e15d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:29.504277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971121bd-479f-4e6f-8848-1a19baa0e15d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:29.505143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971121bd-479f-4e6f-8848-1a19baa0e15d None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:29.506648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971121bd-479f-4e6f-8848-1a19baa0e15d None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:29.507431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971121bd-479f-4e6f-8848-1a19baa0e15d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:29.514725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971121bd-479f-4e6f-8848-1a19baa0e15d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:29.521326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776b17e-8477-451c-84e1-71b7445327d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:29.522467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776b17e-8477-451c-84e1-71b7445327d3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:29.522990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776b17e-8477-451c-84e1-71b7445327d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:29.523498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776b17e-8477-451c-84e1-71b7445327d3 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:29.525534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776b17e-8477-451c-84e1-71b7445327d3 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:29.528495 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776b17e-8477-451c-84e1-71b7445327d3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:29.535208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 192/406] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:29.537098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971121bd-479f-4e6f-8848-1a19baa0e15d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:29.538539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 215/407] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:29 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:29.541365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776b17e-8477-451c-84e1-71b7445327d3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:29.682658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:29.698085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:29.738957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971121bd-479f-4e6f-8848-1a19baa0e15d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:29.753113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776b17e-8477-451c-84e1-71b7445327d3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:29.812590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0b25e-2dbd-4554-bb2e-f8d0fca2815b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:29.814630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0b25e-2dbd-4554-bb2e-f8d0fca2815b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:29.815746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0b25e-2dbd-4554-bb2e-f8d0fca2815b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:29.816762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a0b25e-2dbd-4554-bb2e-f8d0fca2815b None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:29.818388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad64637-cd59-40a8-8e49-165c21eca23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:29.818964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0b25e-2dbd-4554-bb2e-f8d0fca2815b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:29.819483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad64637-cd59-40a8-8e49-165c21eca23c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:29.819924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad64637-cd59-40a8-8e49-165c21eca23c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:29.820361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad64637-cd59-40a8-8e49-165c21eca23c None admin] PATH_INFO: `/v3/groups/5116786fe3844c22b5af4e080a817a3d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:29.820671 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0b25e-2dbd-4554-bb2e-f8d0fca2815b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:29.822563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad64637-cd59-40a8-8e49-165c21eca23c None admin] RBAC: Authorizing `identity:get_group(group_id=5116786fe3844c22b5af4e080a817a3d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:29.824319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a0b25e-2dbd-4554-bb2e-f8d0fca2815b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:29.825108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad64637-cd59-40a8-8e49-165c21eca23c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:29.828597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 216/408] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:29 2022] GET /identity/v3/groups/5116786fe3844c22b5af4e080a817a3d => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:29.830425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad64637-cd59-40a8-8e49-165c21eca23c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:29.842792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 193/409] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:29.849977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0b25e-2dbd-4554-bb2e-f8d0fca2815b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:30.029802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad64637-cd59-40a8-8e49-165c21eca23c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:30.055389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a0b25e-2dbd-4554-bb2e-f8d0fca2815b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:30.102171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d98f01-0178-41c7-99f3-a9f559132ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:30.103555 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d98f01-0178-41c7-99f3-a9f559132ca4 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:30.104551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d98f01-0178-41c7-99f3-a9f559132ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:30.105954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:30.106694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d98f01-0178-41c7-99f3-a9f559132ca4 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9/groups/5116786fe3844c22b5af4e080a817a3d/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:30.110390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d98f01-0178-41c7-99f3-a9f559132ca4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6753f2523be140a3a48348df431cf0d9, group_id=5116786fe3844c22b5af4e080a817a3d, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:30.113260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d98f01-0178-41c7-99f3-a9f559132ca4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:30.131109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b893b-e080-4c85-921e-332e975044a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:30.132720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b893b-e080-4c85-921e-332e975044a4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:30.133455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b893b-e080-4c85-921e-332e975044a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:30.134214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b893b-e080-4c85-921e-332e975044a4 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:30.136775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b893b-e080-4c85-921e-332e975044a4 None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:30.139182 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 217/410] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Sep 21 16:08:29 2022] PUT /identity/v3/projects/6753f2523be140a3a48348df431cf0d9/groups/5116786fe3844c22b5af4e080a817a3d/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:08:30.141101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b893b-e080-4c85-921e-332e975044a4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:30.144979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 194/411] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:29 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:30.146017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d98f01-0178-41c7-99f3-a9f559132ca4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:30.151485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b893b-e080-4c85-921e-332e975044a4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:30.156843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:30.364963 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b893b-e080-4c85-921e-332e975044a4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:30.433529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cd6ed-b4cf-421d-ab50-104d1e1ca012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:30.435302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cd6ed-b4cf-421d-ab50-104d1e1ca012 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:30.436147 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cd6ed-b4cf-421d-ab50-104d1e1ca012 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:30.436913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cd6ed-b4cf-421d-ab50-104d1e1ca012 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800/groups/5116786fe3844c22b5af4e080a817a3d/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:30.440963 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cd6ed-b4cf-421d-ab50-104d1e1ca012 None admin] RBAC: Authorizing `identity:create_grant(project_id=832b79a184af40d7ad323b5e97816800, group_id=5116786fe3844c22b5af4e080a817a3d, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:30.444348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cd6ed-b4cf-421d-ab50-104d1e1ca012 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:30.502408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 195/412] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Sep 21 16:08:30 2022] PUT /identity/v3/projects/832b79a184af40d7ad323b5e97816800/groups/5116786fe3844c22b5af4e080a817a3d/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:08:30.509925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d98f01-0178-41c7-99f3-a9f559132ca4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:30.580318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c933ea-7673-441f-ae56-3ef7679bf54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:30.581266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c933ea-7673-441f-ae56-3ef7679bf54d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:30.581671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c933ea-7673-441f-ae56-3ef7679bf54d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:30.582116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c933ea-7673-441f-ae56-3ef7679bf54d None admin] PATH_INFO: `/v3/groups/d6b80730bdbf48fb93a94fb0197bb9bb` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:30.584295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c933ea-7673-441f-ae56-3ef7679bf54d None admin] RBAC: Authorizing `identity:get_group(group_id=d6b80730bdbf48fb93a94fb0197bb9bb)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:30.587122 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c933ea-7673-441f-ae56-3ef7679bf54d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:30.590907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 218/413] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:30 2022] GET /identity/v3/groups/d6b80730bdbf48fb93a94fb0197bb9bb => generated 229 bytes in 451 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:30.599990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cd6ed-b4cf-421d-ab50-104d1e1ca012 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:30.703825 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:30.806834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cd6ed-b4cf-421d-ab50-104d1e1ca012 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:30.866646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c428bc-3361-49f0-91fe-dc0048a0a90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:30.868954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c428bc-3361-49f0-91fe-dc0048a0a90b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:30.868954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c428bc-3361-49f0-91fe-dc0048a0a90b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:30.868954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c428bc-3361-49f0-91fe-dc0048a0a90b None admin] PATH_INFO: `/v3/projects/92877a675fac490b8473d8157aef4217` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:30.871164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c428bc-3361-49f0-91fe-dc0048a0a90b None admin] RBAC: Authorizing `identity:get_project(project_id=92877a675fac490b8473d8157aef4217)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:30.872763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c428bc-3361-49f0-91fe-dc0048a0a90b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:30.874969 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 196/414] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:30 2022] GET /identity/v3/projects/92877a675fac490b8473d8157aef4217 => generated 344 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:30.881898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c933ea-7673-441f-ae56-3ef7679bf54d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:30.909510 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b2e41-7494-4797-b8b8-7a1025635ed7 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:30.909919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b2e41-7494-4797-b8b8-7a1025635ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:30.910194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851b2e41-7494-4797-b8b8-7a1025635ed7 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:30.911786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 197/415] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:30.919702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b67fc-beb9-4aaa-a09c-91f44540a92d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:30.920055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b67fc-beb9-4aaa-a09c-91f44540a92d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:30.920381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b67fc-beb9-4aaa-a09c-91f44540a92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:30.946197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-080b67fc-beb9-4aaa-a09c-91f44540a92d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:30.947853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-080b67fc-beb9-4aaa-a09c-91f44540a92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:31.031294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:31.083763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c933ea-7673-441f-ae56-3ef7679bf54d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:31.089844 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 198/416] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:08:31.165571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a144eb-1499-45f9-be8f-072edfcf4724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:31.166734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a144eb-1499-45f9-be8f-072edfcf4724 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:31.167221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a144eb-1499-45f9-be8f-072edfcf4724 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:31.167774 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a144eb-1499-45f9-be8f-072edfcf4724 None admin] PATH_INFO: `/v3/projects/92877a675fac490b8473d8157aef4217/groups/d6b80730bdbf48fb93a94fb0197bb9bb/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:31.171379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a144eb-1499-45f9-be8f-072edfcf4724 None admin] RBAC: Authorizing `identity:create_grant(project_id=92877a675fac490b8473d8157aef4217, group_id=d6b80730bdbf48fb93a94fb0197bb9bb, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:31.173966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a144eb-1499-45f9-be8f-072edfcf4724 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:31.204571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 219/417] 10.222.0.64 () {58 vars in 1442 bytes} [Wed Sep 21 16:08:30 2022] PUT /identity/v3/projects/92877a675fac490b8473d8157aef4217/groups/d6b80730bdbf48fb93a94fb0197bb9bb/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:08:31.596270 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b67fc-beb9-4aaa-a09c-91f44540a92d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:31.632424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58112c-dfff-4c6b-8001-8e55c5025d81 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:31.633857 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58112c-dfff-4c6b-8001-8e55c5025d81 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:31.634838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58112c-dfff-4c6b-8001-8e55c5025d81 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:31.639476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 220/418] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:31.648603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0b174-c8ee-4403-b188-6da7c7b358f2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:31.649527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0b174-c8ee-4403-b188-6da7c7b358f2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:31.650285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0b174-c8ee-4403-b188-6da7c7b358f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:31.707417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-50a0b174-c8ee-4403-b188-6da7c7b358f2 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:31.710679 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-50a0b174-c8ee-4403-b188-6da7c7b358f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:31.738517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:31.844583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b67fc-beb9-4aaa-a09c-91f44540a92d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:31.924379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 221/419] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:31.960478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb285385-4b3b-4cff-bd2c-31454a2d7a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:31.961428 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb285385-4b3b-4cff-bd2c-31454a2d7a18 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:31.961918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb285385-4b3b-4cff-bd2c-31454a2d7a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:31.962438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb285385-4b3b-4cff-bd2c-31454a2d7a18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:31.989961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb285385-4b3b-4cff-bd2c-31454a2d7a18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-eb285385-4b3b-4cff-bd2c-31454a2d7a18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:32.003764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:32.004708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:32.005971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 199/420] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:08:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:32.011584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0b174-c8ee-4403-b188-6da7c7b358f2 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:32.112795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:32.167608 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0b174-c8ee-4403-b188-6da7c7b358f2 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:32.205257 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5bc3d-e8f2-4446-9b89-6cb44fea7081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:32.205963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5bc3d-e8f2-4446-9b89-6cb44fea7081 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:32.206283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5bc3d-e8f2-4446-9b89-6cb44fea7081 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:32.206556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5bc3d-e8f2-4446-9b89-6cb44fea7081 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:32.207265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5bc3d-e8f2-4446-9b89-6cb44fea7081 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:32.207575 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5bc3d-e8f2-4446-9b89-6cb44fea7081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:32.209036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5bc3d-e8f2-4446-9b89-6cb44fea7081 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:32.219768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 222/421] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:32.226546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb285385-4b3b-4cff-bd2c-31454a2d7a18 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:32.387310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5bc3d-e8f2-4446-9b89-6cb44fea7081 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:32.423187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb285385-4b3b-4cff-bd2c-31454a2d7a18 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:32.435409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:32.509762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca070ed-da41-4def-8f27-09d1ca74add3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:32.509762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca070ed-da41-4def-8f27-09d1ca74add3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:32.509762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca070ed-da41-4def-8f27-09d1ca74add3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:32.509762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca070ed-da41-4def-8f27-09d1ca74add3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:32.593562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca070ed-da41-4def-8f27-09d1ca74add3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:32.595558 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca070ed-da41-4def-8f27-09d1ca74add3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:32.630862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5bc3d-e8f2-4446-9b89-6cb44fea7081 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:32.694360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:32.730914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23fc3c-ac7b-4533-814a-a2074eccdf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:32.731603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23fc3c-ac7b-4533-814a-a2074eccdf67 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:32.731890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23fc3c-ac7b-4533-814a-a2074eccdf67 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:32.732170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23fc3c-ac7b-4533-814a-a2074eccdf67 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:32.732922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 200/422] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 16:08:32.743142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca070ed-da41-4def-8f27-09d1ca74add3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:32.756447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23fc3c-ac7b-4533-814a-a2074eccdf67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:32.761982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-2f23fc3c-ac7b-4533-814a-a2074eccdf67 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:32.764843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:32.766623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:32.768539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 223/423] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:08:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:32.783162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23fc3c-ac7b-4533-814a-a2074eccdf67 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:32.906688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca070ed-da41-4def-8f27-09d1ca74add3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:32.971766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c498e10-ecf1-45ca-86c0-26bdaa79e109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:32.972338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c498e10-ecf1-45ca-86c0-26bdaa79e109 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:32.972613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c498e10-ecf1-45ca-86c0-26bdaa79e109 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:32.972883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c498e10-ecf1-45ca-86c0-26bdaa79e109 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:32.975213 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c498e10-ecf1-45ca-86c0-26bdaa79e109 None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:32.976690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c498e10-ecf1-45ca-86c0-26bdaa79e109 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:32.978826 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 201/424] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:32 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:32.991558 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c498e10-ecf1-45ca-86c0-26bdaa79e109 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:33.089409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23fc3c-ac7b-4533-814a-a2074eccdf67 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:33.136054 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27912f-f7e8-4dab-a8ab-e88a7baf81a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:33.137196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27912f-f7e8-4dab-a8ab-e88a7baf81a8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:33.137476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27912f-f7e8-4dab-a8ab-e88a7baf81a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:33.137748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27912f-f7e8-4dab-a8ab-e88a7baf81a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:33.138459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27912f-f7e8-4dab-a8ab-e88a7baf81a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:33.138796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27912f-f7e8-4dab-a8ab-e88a7baf81a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:33.140187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f27912f-f7e8-4dab-a8ab-e88a7baf81a8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:33.141627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:33.149359 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 224/425] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:33.156643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27912f-f7e8-4dab-a8ab-e88a7baf81a8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:33.191491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c498e10-ecf1-45ca-86c0-26bdaa79e109 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:33.245176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad518b1-aae7-4f55-8fd8-330203378467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:33.247610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad518b1-aae7-4f55-8fd8-330203378467 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:33.248655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad518b1-aae7-4f55-8fd8-330203378467 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:33.249593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad518b1-aae7-4f55-8fd8-330203378467 None admin] PATH_INFO: `/v3/groups/5116786fe3844c22b5af4e080a817a3d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:33.252088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad518b1-aae7-4f55-8fd8-330203378467 None admin] RBAC: Authorizing `identity:get_group(group_id=5116786fe3844c22b5af4e080a817a3d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:33.255155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad518b1-aae7-4f55-8fd8-330203378467 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:33.259531 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 202/426] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:32 2022] GET /identity/v3/groups/5116786fe3844c22b5af4e080a817a3d => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:33.270317 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad518b1-aae7-4f55-8fd8-330203378467 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:33.322622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:33.518186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f27912f-f7e8-4dab-a8ab-e88a7baf81a8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:33.576778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad518b1-aae7-4f55-8fd8-330203378467 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:33.581574 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e37ee-9491-4085-8e4c-b0050b9d36ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:33.582233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e37ee-9491-4085-8e4c-b0050b9d36ee None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:33.582535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e37ee-9491-4085-8e4c-b0050b9d36ee None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:33.582808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e37ee-9491-4085-8e4c-b0050b9d36ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:33.652357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b58a31-b959-4279-8202-5db10e2d76ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:33.652978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b58a31-b959-4279-8202-5db10e2d76ac None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:33.653258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b58a31-b959-4279-8202-5db10e2d76ac None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:33.653565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b58a31-b959-4279-8202-5db10e2d76ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:33.654421 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b58a31-b959-4279-8202-5db10e2d76ac None admin] RBAC: Adding query filter params (group.id=5116786fe3844c22b5af4e080a817a3d, role.id=aa320f864d914f65bebe5463994ff536, scope.project.id=832b79a184af40d7ad323b5e97816800) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:33.654730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b58a31-b959-4279-8202-5db10e2d76ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:33.656426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b58a31-b959-4279-8202-5db10e2d76ac None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:33.673614 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 203/427] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Sep 21 16:08:33 2022] GET /identity/v3/role_assignments?group.id=5116786fe3844c22b5af4e080a817a3d&scope.project.id=832b79a184af40d7ad323b5e97816800&role.id=aa320f864d914f65bebe5463994ff536 => generated 611 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:33.728410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e37ee-9491-4085-8e4c-b0050b9d36ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:33.731922 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e37ee-9491-4085-8e4c-b0050b9d36ee None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:33.836211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:33.933317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 225/428] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 777 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:33.941541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b58a31-b959-4279-8202-5db10e2d76ac None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:33.968094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:34.077686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a94695-09ba-4db5-b6cd-9d50c6a4044c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:34.078232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a94695-09ba-4db5-b6cd-9d50c6a4044c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:34.078723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a94695-09ba-4db5-b6cd-9d50c6a4044c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:34.081298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 226/429] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:34.091175 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584eb62-57b6-41dd-a75e-511c6206966c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:34.091593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584eb62-57b6-41dd-a75e-511c6206966c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:34.092036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6584eb62-57b6-41dd-a75e-511c6206966c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:34.143160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6584eb62-57b6-41dd-a75e-511c6206966c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:34.145209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6584eb62-57b6-41dd-a75e-511c6206966c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:34.164792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b58a31-b959-4279-8202-5db10e2d76ac None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:34.242270 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26fa70-8d78-4d1f-9faa-e671fb76ae9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:34.243707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26fa70-8d78-4d1f-9faa-e671fb76ae9c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:34.244588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26fa70-8d78-4d1f-9faa-e671fb76ae9c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:34.245139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26fa70-8d78-4d1f-9faa-e671fb76ae9c None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:34.247232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26fa70-8d78-4d1f-9faa-e671fb76ae9c None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:34.248837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26fa70-8d78-4d1f-9faa-e671fb76ae9c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:34.251137 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 204/430] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:33 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:34.256780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26fa70-8d78-4d1f-9faa-e671fb76ae9c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:34.330127 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 227/431] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:34.378801 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:34.405270 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26fa70-8d78-4d1f-9faa-e671fb76ae9c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:34.417940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44cb77-b279-4466-a2af-5acde9fd020e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:34.418282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44cb77-b279-4466-a2af-5acde9fd020e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:34.418662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44cb77-b279-4466-a2af-5acde9fd020e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:34.421112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 228/432] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:08:34.427150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe497a-0242-4ae8-a8aa-76248f8f9fc7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:34.427739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe497a-0242-4ae8-a8aa-76248f8f9fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:34.428244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe497a-0242-4ae8-a8aa-76248f8f9fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:34.448198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18376277-a278-48f5-a74b-ab394cb3ba58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:34.448887 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18376277-a278-48f5-a74b-ab394cb3ba58 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:34.449172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18376277-a278-48f5-a74b-ab394cb3ba58 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:34.449407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0fbe497a-0242-4ae8-a8aa-76248f8f9fc7 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:34.449508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18376277-a278-48f5-a74b-ab394cb3ba58 None admin] PATH_INFO: `/v3/groups/5116786fe3844c22b5af4e080a817a3d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:34.450941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18376277-a278-48f5-a74b-ab394cb3ba58 None admin] RBAC: Authorizing `identity:get_group(group_id=5116786fe3844c22b5af4e080a817a3d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:34.451574 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0fbe497a-0242-4ae8-a8aa-76248f8f9fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:34.452480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18376277-a278-48f5-a74b-ab394cb3ba58 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:34.454448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 205/433] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:34 2022] GET /identity/v3/groups/5116786fe3844c22b5af4e080a817a3d => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:34.460301 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18376277-a278-48f5-a74b-ab394cb3ba58 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:34.536602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 229/434] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:08:34.537922 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ca5e4-7bbf-4b7c-b35d-7f6ec52a6a30 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:34.538252 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ca5e4-7bbf-4b7c-b35d-7f6ec52a6a30 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:34.538532 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ca5e4-7bbf-4b7c-b35d-7f6ec52a6a30 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:34.542097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 230/435] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:34.546022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768cd0c-0fa8-4104-ae7b-c8da8d156d60 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:34.546346 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768cd0c-0fa8-4104-ae7b-c8da8d156d60 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:34.546628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768cd0c-0fa8-4104-ae7b-c8da8d156d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:34.562170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:34.569572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0768cd0c-0fa8-4104-ae7b-c8da8d156d60 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:34.570867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0768cd0c-0fa8-4104-ae7b-c8da8d156d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:34.603272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18376277-a278-48f5-a74b-ab394cb3ba58 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:34.639597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e45945-a119-482e-9c48-6ed2dbcabf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:34.640154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e45945-a119-482e-9c48-6ed2dbcabf24 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:34.640438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e45945-a119-482e-9c48-6ed2dbcabf24 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:34.640715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e45945-a119-482e-9c48-6ed2dbcabf24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:34.641461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e45945-a119-482e-9c48-6ed2dbcabf24 None admin] RBAC: Adding query filter params (group.id=5116786fe3844c22b5af4e080a817a3d, role.id=aa320f864d914f65bebe5463994ff536, scope.project.id=6753f2523be140a3a48348df431cf0d9) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:34.641777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e45945-a119-482e-9c48-6ed2dbcabf24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:34.643274 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e45945-a119-482e-9c48-6ed2dbcabf24 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:34.657952 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 206/436] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Sep 21 16:08:34 2022] GET /identity/v3/role_assignments?group.id=5116786fe3844c22b5af4e080a817a3d&scope.project.id=6753f2523be140a3a48348df431cf0d9&role.id=aa320f864d914f65bebe5463994ff536 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:34.666585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 231/437] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:08:34.860101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768cd0c-0fa8-4104-ae7b-c8da8d156d60 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:34.860557 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e45945-a119-482e-9c48-6ed2dbcabf24 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:34.879548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:34.996146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e45945-a119-482e-9c48-6ed2dbcabf24 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:35.018236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768cd0c-0fa8-4104-ae7b-c8da8d156d60 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:35.034812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69090a76-8e19-4d77-baaf-c4704dc94ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:35.035456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69090a76-8e19-4d77-baaf-c4704dc94ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:35.035737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69090a76-8e19-4d77-baaf-c4704dc94ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:35.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69090a76-8e19-4d77-baaf-c4704dc94ba9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:35.047445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69090a76-8e19-4d77-baaf-c4704dc94ba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-69090a76-8e19-4d77-baaf-c4704dc94ba9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:35.052889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:35.054546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 207/438] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:35.055441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69090a76-8e19-4d77-baaf-c4704dc94ba9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:35.061144 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1949ce-d8bf-48f8-89bb-9e9ecd99101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:35.061802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1949ce-d8bf-48f8-89bb-9e9ecd99101c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:35.062113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1949ce-d8bf-48f8-89bb-9e9ecd99101c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:35.062388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1949ce-d8bf-48f8-89bb-9e9ecd99101c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:35.065527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:35.075684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1949ce-d8bf-48f8-89bb-9e9ecd99101c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-0b1949ce-d8bf-48f8-89bb-9e9ecd99101c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:35.080624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:35.081795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 232/439] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:08:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:35.082706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1949ce-d8bf-48f8-89bb-9e9ecd99101c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:35.134147 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:35.174388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69090a76-8e19-4d77-baaf-c4704dc94ba9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:35.192848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1949ce-d8bf-48f8-89bb-9e9ecd99101c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:35.216471 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5da49-5cd1-4e97-9aab-965917edb8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:35.217287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5da49-5cd1-4e97-9aab-965917edb8bc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:35.217660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5da49-5cd1-4e97-9aab-965917edb8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:35.218067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5da49-5cd1-4e97-9aab-965917edb8bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:35.231494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5da49-5cd1-4e97-9aab-965917edb8bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:35.235042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ae14c-9b7c-4384-b410-8523e380acb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:35.235665 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ae14c-9b7c-4384-b410-8523e380acb5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:35.235947 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ae14c-9b7c-4384-b410-8523e380acb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:35.236223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ae14c-9b7c-4384-b410-8523e380acb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:35.236908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ae14c-9b7c-4384-b410-8523e380acb5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-97d5da49-5cd1-4e97-9aab-965917edb8bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:35.237161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:35.239037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:35.241464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:35.241464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:35.241464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:35.241464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:35.241464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:35.241464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:35.241464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:35.241464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:35.241464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:35.241464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:35.241464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 21 16:08:35.241464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:35.242947 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ae14c-9b7c-4384-b410-8523e380acb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:35.243093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 208/440] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:35.243234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ae14c-9b7c-4384-b410-8523e380acb5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:35.243363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5da49-5cd1-4e97-9aab-965917edb8bc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:35.249245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 233/441] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:35.250069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ae14c-9b7c-4384-b410-8523e380acb5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:35.332063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:35.391368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ae14c-9b7c-4384-b410-8523e380acb5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:35.431096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032c086-c0e8-439c-8809-19432da22932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:35.431852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032c086-c0e8-439c-8809-19432da22932 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:35.432230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032c086-c0e8-439c-8809-19432da22932 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:35.432602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032c086-c0e8-439c-8809-19432da22932 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:35.433967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032c086-c0e8-439c-8809-19432da22932 None admin] RBAC: Adding query filter params (name=member) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:35.434412 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032c086-c0e8-439c-8809-19432da22932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:35.436132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032c086-c0e8-439c-8809-19432da22932 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:35.446238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 234/442] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:08:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:35.447714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032c086-c0e8-439c-8809-19432da22932 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:35.469348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5da49-5cd1-4e97-9aab-965917edb8bc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:35.520715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:35.534892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bf14f-299a-4c51-b8d6-cc10bc7962a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:35.536025 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bf14f-299a-4c51-b8d6-cc10bc7962a4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:35.536221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bf14f-299a-4c51-b8d6-cc10bc7962a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:35.536495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05bf14f-299a-4c51-b8d6-cc10bc7962a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:35.537427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bf14f-299a-4c51-b8d6-cc10bc7962a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:35.537851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bf14f-299a-4c51-b8d6-cc10bc7962a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:35.539930 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05bf14f-299a-4c51-b8d6-cc10bc7962a4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:35.553163 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 209/443] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:35.554375 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bf14f-299a-4c51-b8d6-cc10bc7962a4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:35.586717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032c086-c0e8-439c-8809-19432da22932 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:35.626552 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:35.650070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f813ea3-6c98-4347-b799-e39bdd1b62b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:35.651150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f813ea3-6c98-4347-b799-e39bdd1b62b0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:35.652203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f813ea3-6c98-4347-b799-e39bdd1b62b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:35.652698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f813ea3-6c98-4347-b799-e39bdd1b62b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:35.707878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05bf14f-299a-4c51-b8d6-cc10bc7962a4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:35.766926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f813ea3-6c98-4347-b799-e39bdd1b62b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:35.767689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a07ef2-6a9b-4d1b-91f2-947b5d53241d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:35.768578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a07ef2-6a9b-4d1b-91f2-947b5d53241d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:35.768989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a07ef2-6a9b-4d1b-91f2-947b5d53241d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:35.769404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a07ef2-6a9b-4d1b-91f2-947b5d53241d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:35.769545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f813ea3-6c98-4347-b799-e39bdd1b62b0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:35.863660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:35.879306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a07ef2-6a9b-4d1b-91f2-947b5d53241d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:35.881497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a07ef2-6a9b-4d1b-91f2-947b5d53241d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:35.926069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 235/444] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:35.927146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f813ea3-6c98-4347-b799-e39bdd1b62b0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:35.954468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:36.056024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 210/445] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:36.057052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a07ef2-6a9b-4d1b-91f2-947b5d53241d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:36.107799 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f813ea3-6c98-4347-b799-e39bdd1b62b0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:36.151988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a4a6-8081-427b-940e-c08322ee603c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:36.152582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a4a6-8081-427b-940e-c08322ee603c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:36.152863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a4a6-8081-427b-940e-c08322ee603c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:36.153129 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a4a6-8081-427b-940e-c08322ee603c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:36.211106 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:36.228441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a07ef2-6a9b-4d1b-91f2-947b5d53241d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:36.243893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4a4a6-8081-427b-940e-c08322ee603c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:36.245621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4a4a6-8081-427b-940e-c08322ee603c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:36.271928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:36.284086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cbc2f-97cc-4dbf-86ac-b25ef171fc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:36.284961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cbc2f-97cc-4dbf-86ac-b25ef171fc52 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:36.285341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cbc2f-97cc-4dbf-86ac-b25ef171fc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:36.285717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cbc2f-97cc-4dbf-86ac-b25ef171fc52 None admin] PATH_INFO: `/v3/projects/6753f2523be140a3a48348df431cf0d9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:36.287472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cbc2f-97cc-4dbf-86ac-b25ef171fc52 None admin] RBAC: Authorizing `identity:get_project(project_id=6753f2523be140a3a48348df431cf0d9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:36.289630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324cbc2f-97cc-4dbf-86ac-b25ef171fc52 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:36.292661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 211/446] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:36 2022] GET /identity/v3/projects/6753f2523be140a3a48348df431cf0d9 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:36.293770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cbc2f-97cc-4dbf-86ac-b25ef171fc52 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:36.397217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 236/447] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:36.398117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a4a6-8081-427b-940e-c08322ee603c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:36.426321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cbc2f-97cc-4dbf-86ac-b25ef171fc52 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:36.482378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c93e7-e2d7-438f-ab5b-63ae08c66a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:36.483323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c93e7-e2d7-438f-ab5b-63ae08c66a63 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:36.483749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c93e7-e2d7-438f-ab5b-63ae08c66a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:36.484242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c93e7-e2d7-438f-ab5b-63ae08c66a63 None admin] PATH_INFO: `/v3/projects/832b79a184af40d7ad323b5e97816800` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:36.486357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c93e7-e2d7-438f-ab5b-63ae08c66a63 None admin] RBAC: Authorizing `identity:get_project(project_id=832b79a184af40d7ad323b5e97816800)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:36.489103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c93e7-e2d7-438f-ab5b-63ae08c66a63 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:36.492125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 212/448] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:36 2022] GET /identity/v3/projects/832b79a184af40d7ad323b5e97816800 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:36.493540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c93e7-e2d7-438f-ab5b-63ae08c66a63 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:36.516966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a4a6-8081-427b-940e-c08322ee603c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:36.539819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:36.540693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:36.556769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504440cf-d5ca-4eb7-8c79-d0ce2f3668cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:36.557413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504440cf-d5ca-4eb7-8c79-d0ce2f3668cc None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:36.557714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504440cf-d5ca-4eb7-8c79-d0ce2f3668cc None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:36.557994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504440cf-d5ca-4eb7-8c79-d0ce2f3668cc None admin] PATH_INFO: `/v3/groups/5116786fe3844c22b5af4e080a817a3d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:36.559199 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504440cf-d5ca-4eb7-8c79-d0ce2f3668cc None admin] RBAC: Authorizing `identity:get_group(group_id=5116786fe3844c22b5af4e080a817a3d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:36.560665 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504440cf-d5ca-4eb7-8c79-d0ce2f3668cc None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:36.562316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 237/449] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:36 2022] GET /identity/v3/groups/5116786fe3844c22b5af4e080a817a3d => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:36.563099 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504440cf-d5ca-4eb7-8c79-d0ce2f3668cc None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:36.645250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c93e7-e2d7-438f-ab5b-63ae08c66a63 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:36.683436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574100e-10bb-4630-a771-bea2b6925a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:36.683991 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574100e-10bb-4630-a771-bea2b6925a31 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:36.684273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574100e-10bb-4630-a771-bea2b6925a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:36.684550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574100e-10bb-4630-a771-bea2b6925a31 None admin] PATH_INFO: `/v3/projects/92877a675fac490b8473d8157aef4217` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:36.685729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574100e-10bb-4630-a771-bea2b6925a31 None admin] RBAC: Authorizing `identity:get_project(project_id=92877a675fac490b8473d8157aef4217)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:36.686089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504440cf-d5ca-4eb7-8c79-d0ce2f3668cc None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:36.687226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7574100e-10bb-4630-a771-bea2b6925a31 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:36.688917 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 213/450] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:08:36 2022] GET /identity/v3/projects/92877a675fac490b8473d8157aef4217 => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:36.689715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574100e-10bb-4630-a771-bea2b6925a31 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:36.736774 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758ec95-d9a9-4cdb-a269-b2ea3e101143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:36.737786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758ec95-d9a9-4cdb-a269-b2ea3e101143 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:36.738292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758ec95-d9a9-4cdb-a269-b2ea3e101143 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:36.738748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758ec95-d9a9-4cdb-a269-b2ea3e101143 None admin] PATH_INFO: `/v3/groups/5116786fe3844c22b5af4e080a817a3d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:36.740511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758ec95-d9a9-4cdb-a269-b2ea3e101143 None admin] RBAC: Authorizing `identity:get_group(group_id=5116786fe3844c22b5af4e080a817a3d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:36.742904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758ec95-d9a9-4cdb-a269-b2ea3e101143 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:36.751179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 238/451] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:36 2022] GET /identity/v3/groups/5116786fe3844c22b5af4e080a817a3d => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:36.751179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758ec95-d9a9-4cdb-a269-b2ea3e101143 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:36.822155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:36.827596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:36.832345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7574100e-10bb-4630-a771-bea2b6925a31 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:36.871166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090abbd-0cd7-4752-8cf2-ae894f910b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:36.871777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090abbd-0cd7-4752-8cf2-ae894f910b68 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:36.872067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090abbd-0cd7-4752-8cf2-ae894f910b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:36.872344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090abbd-0cd7-4752-8cf2-ae894f910b68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:36.873497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090abbd-0cd7-4752-8cf2-ae894f910b68 None admin] RBAC: Adding query filter params (group.id=5116786fe3844c22b5af4e080a817a3d, role.id=20378c15bf524a4c960b49d2c53cf6b2, scope.project.id=6753f2523be140a3a48348df431cf0d9) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:36.873497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090abbd-0cd7-4752-8cf2-ae894f910b68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:36.874882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090abbd-0cd7-4752-8cf2-ae894f910b68 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:36.881486 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 214/452] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Sep 21 16:08:36 2022] GET /identity/v3/role_assignments?group.id=5116786fe3844c22b5af4e080a817a3d&scope.project.id=6753f2523be140a3a48348df431cf0d9&role.id=20378c15bf524a4c960b49d2c53cf6b2 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:36.882349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090abbd-0cd7-4752-8cf2-ae894f910b68 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:36.895420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758ec95-d9a9-4cdb-a269-b2ea3e101143 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:36.947008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda963c-89b6-4d89-ad5d-49bdd3e844a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:36.947886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda963c-89b6-4d89-ad5d-49bdd3e844a7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:36.948899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda963c-89b6-4d89-ad5d-49bdd3e844a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:36.949361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eda963c-89b6-4d89-ad5d-49bdd3e844a7 None admin] PATH_INFO: `/v3/groups/d6b80730bdbf48fb93a94fb0197bb9bb` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:36.951582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda963c-89b6-4d89-ad5d-49bdd3e844a7 None admin] RBAC: Authorizing `identity:get_group(group_id=d6b80730bdbf48fb93a94fb0197bb9bb)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:36.954160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eda963c-89b6-4d89-ad5d-49bdd3e844a7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:36.956965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 239/453] 10.222.0.64 () {56 vars in 1180 bytes} [Wed Sep 21 16:08:36 2022] GET /identity/v3/groups/d6b80730bdbf48fb93a94fb0197bb9bb => generated 229 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:36.963957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda963c-89b6-4d89-ad5d-49bdd3e844a7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:37.053910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090abbd-0cd7-4752-8cf2-ae894f910b68 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:37.108381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff3b26c-c54f-4a94-b7bf-4ef0f98e728b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:37.109364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3b26c-c54f-4a94-b7bf-4ef0f98e728b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:37.109815 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3b26c-c54f-4a94-b7bf-4ef0f98e728b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:37.110312 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff3b26c-c54f-4a94-b7bf-4ef0f98e728b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:37.111629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3b26c-c54f-4a94-b7bf-4ef0f98e728b None admin] RBAC: Adding query filter params (group.id=5116786fe3844c22b5af4e080a817a3d, role.id=20378c15bf524a4c960b49d2c53cf6b2, scope.project.id=832b79a184af40d7ad323b5e97816800) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:37.112140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3b26c-c54f-4a94-b7bf-4ef0f98e728b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:37.114873 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff3b26c-c54f-4a94-b7bf-4ef0f98e728b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:37.127186 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 215/454] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Sep 21 16:08:36 2022] GET /identity/v3/role_assignments?group.id=5116786fe3844c22b5af4e080a817a3d&scope.project.id=832b79a184af40d7ad323b5e97816800&role.id=20378c15bf524a4c960b49d2c53cf6b2 => generated 611 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:37.147762 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eda963c-89b6-4d89-ad5d-49bdd3e844a7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:37.195633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:37.214779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23d6ba-478b-4633-9f41-f8feb2a70b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:37.215405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23d6ba-478b-4633-9f41-f8feb2a70b67 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:37.215742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23d6ba-478b-4633-9f41-f8feb2a70b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:37.216072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23d6ba-478b-4633-9f41-f8feb2a70b67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:37.216911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23d6ba-478b-4633-9f41-f8feb2a70b67 None admin] RBAC: Adding query filter params (group.id=d6b80730bdbf48fb93a94fb0197bb9bb, role.id=459d01a218314543a7a8329e88b78331, scope.project.id=92877a675fac490b8473d8157aef4217) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:37.217279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23d6ba-478b-4633-9f41-f8feb2a70b67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:37.219434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23d6ba-478b-4633-9f41-f8feb2a70b67 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:37.229253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 240/455] 10.222.0.64 () {56 vars in 1376 bytes} [Wed Sep 21 16:08:36 2022] GET /identity/v3/role_assignments?group.id=d6b80730bdbf48fb93a94fb0197bb9bb&scope.project.id=92877a675fac490b8473d8157aef4217&role.id=459d01a218314543a7a8329e88b78331 => generated 611 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:40.511690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d0e8c-0d6b-4d66-9c1b-026a77e6b152 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:40.512648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d0e8c-0d6b-4d66-9c1b-026a77e6b152 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:40.513152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d0e8c-0d6b-4d66-9c1b-026a77e6b152 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:40.515939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 216/456] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:40.520560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d201d-42f1-4eef-8ef5-a7d445b847ca None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:40.521059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d201d-42f1-4eef-8ef5-a7d445b847ca None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:40.521135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d201d-42f1-4eef-8ef5-a7d445b847ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:40.547138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a60d201d-42f1-4eef-8ef5-a7d445b847ca None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:40.548356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a60d201d-42f1-4eef-8ef5-a7d445b847ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:40.642075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 241/457] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:41.052440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266662-b89c-405f-8e32-2cd3ce8744a9 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:41.053149 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266662-b89c-405f-8e32-2cd3ce8744a9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:41.053716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72266662-b89c-405f-8e32-2cd3ce8744a9 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:41.055392 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 217/458] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:41.065619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e13a5-ea26-4783-b525-9e1d19329912 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:41.065619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e13a5-ea26-4783-b525-9e1d19329912 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:41.065619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e13a5-ea26-4783-b525-9e1d19329912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:41.092043 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c96e13a5-ea26-4783-b525-9e1d19329912 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:41.094282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c96e13a5-ea26-4783-b525-9e1d19329912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:41.202702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72266662-b89c-405f-8e32-2cd3ce8744a9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:41.236899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 242/459] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:41.239135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03018f4b-8d51-424a-9218-107d7c47a43d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:41.239624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03018f4b-8d51-424a-9218-107d7c47a43d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:41.240118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03018f4b-8d51-424a-9218-107d7c47a43d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:41.242907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 243/460] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:41.250606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:41.253976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51924ba6-bba3-4049-83da-ed21f836348f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:41.254523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51924ba6-bba3-4049-83da-ed21f836348f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:41.254973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51924ba6-bba3-4049-83da-ed21f836348f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:41.286372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-51924ba6-bba3-4049-83da-ed21f836348f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:41.288647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-51924ba6-bba3-4049-83da-ed21f836348f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:41.341424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72266662-b89c-405f-8e32-2cd3ce8744a9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:41.393476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:41.395716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3caaa52-52c3-42c1-8e03-87fb9d6f6634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:41.396437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caaa52-52c3-42c1-8e03-87fb9d6f6634 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:41.396719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caaa52-52c3-42c1-8e03-87fb9d6f6634 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:41.397076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caaa52-52c3-42c1-8e03-87fb9d6f6634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:41.426318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3caaa52-52c3-42c1-8e03-87fb9d6f6634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:41.427786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3caaa52-52c3-42c1-8e03-87fb9d6f6634 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:41.453333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 244/461] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d3caaa52-52c3-42c1-8e03-87fb9d6f6634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:41.463340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:41.464427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:41.465265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:41.465265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:41.465265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:41.465265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 218/462] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:41.471896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51924ba6-bba3-4049-83da-ed21f836348f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:41.525595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099506f-5efa-458d-83b2-202d4df313e9 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:41.526172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099506f-5efa-458d-83b2-202d4df313e9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:41.526478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099506f-5efa-458d-83b2-202d4df313e9 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:41.528314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 219/463] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:08:41.533454 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce35160-58f7-4c49-a343-9910027b179e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:41.534015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce35160-58f7-4c49-a343-9910027b179e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:41.534509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce35160-58f7-4c49-a343-9910027b179e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:41.575357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8ce35160-58f7-4c49-a343-9910027b179e None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:41.577020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8ce35160-58f7-4c49-a343-9910027b179e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:41.598087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51924ba6-bba3-4049-83da-ed21f836348f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:41.643400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cbdd7-ea58-4638-ac42-41558c801aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:41.644268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cbdd7-ea58-4638-ac42-41558c801aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:41.644692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cbdd7-ea58-4638-ac42-41558c801aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:41.645138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cbdd7-ea58-4638-ac42-41558c801aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:41.645934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cbdd7-ea58-4638-ac42-41558c801aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:41.646401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cbdd7-ea58-4638-ac42-41558c801aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:41.647954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cbdd7-ea58-4638-ac42-41558c801aa2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:41.680779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 220/464] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:41.684228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 245/465] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:41.689889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce35160-58f7-4c49-a343-9910027b179e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:41.694580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:41.801972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cbdd7-ea58-4638-ac42-41558c801aa2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:41.831717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce35160-58f7-4c49-a343-9910027b179e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:41.869547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e5ac6-ae04-46d4-8fae-4ee6777096fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:41.870680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e5ac6-ae04-46d4-8fae-4ee6777096fd None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:41.870960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e5ac6-ae04-46d4-8fae-4ee6777096fd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:41.871232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e5ac6-ae04-46d4-8fae-4ee6777096fd None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:41.871905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e5ac6-ae04-46d4-8fae-4ee6777096fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:41.873350 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e5ac6-ae04-46d4-8fae-4ee6777096fd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:41.897060 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 221/466] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:08:41 2022] POST /identity/v3/users => generated 246 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:08:41.897975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e5ac6-ae04-46d4-8fae-4ee6777096fd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:41.918407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:41.999792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:42.055091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e5ac6-ae04-46d4-8fae-4ee6777096fd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:42.095266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d355d4e-a9fd-473e-8c96-06e3e9959d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:42.095914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d355d4e-a9fd-473e-8c96-06e3e9959d05 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:42.096194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d355d4e-a9fd-473e-8c96-06e3e9959d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:42.096466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d355d4e-a9fd-473e-8c96-06e3e9959d05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:42.098072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cbdd7-ea58-4638-ac42-41558c801aa2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:42.109203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d355d4e-a9fd-473e-8c96-06e3e9959d05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:42.110739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d355d4e-a9fd-473e-8c96-06e3e9959d05 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-6d355d4e-a9fd-473e-8c96-06e3e9959d05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:42.127655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:42.128588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:42.129427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:42.129427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:42.129427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:42.129427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 222/467] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:42.139687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d355d4e-a9fd-473e-8c96-06e3e9959d05 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:42.171441 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf62424-6bd6-4960-8bab-2eee9b064f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:42.172415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf62424-6bd6-4960-8bab-2eee9b064f30 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:42.172759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf62424-6bd6-4960-8bab-2eee9b064f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:42.173040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf62424-6bd6-4960-8bab-2eee9b064f30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:42.187105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf62424-6bd6-4960-8bab-2eee9b064f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:42.188576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf62424-6bd6-4960-8bab-2eee9b064f30 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6cf62424-6bd6-4960-8bab-2eee9b064f30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:42.210886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:42.211971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:42.212925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:42.212925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:42.212925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:42.212925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 246/468] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:42.212925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf62424-6bd6-4960-8bab-2eee9b064f30 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:42.318097 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d355d4e-a9fd-473e-8c96-06e3e9959d05 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:42.335958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:42.347369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:42.361750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf62424-6bd6-4960-8bab-2eee9b064f30 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:42.372574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2af094-605f-42b5-82d3-96a8fab220c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:42.373194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2af094-605f-42b5-82d3-96a8fab220c0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:42.373478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2af094-605f-42b5-82d3-96a8fab220c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:42.373755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2af094-605f-42b5-82d3-96a8fab220c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:42.374457 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2af094-605f-42b5-82d3-96a8fab220c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:42.374832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2af094-605f-42b5-82d3-96a8fab220c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:42.377037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2af094-605f-42b5-82d3-96a8fab220c0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:42.395232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 223/469] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:42.396206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2af094-605f-42b5-82d3-96a8fab220c0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:42.416630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b80dd-0816-46f6-a1ec-113602eab8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:42.417303 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b80dd-0816-46f6-a1ec-113602eab8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:42.417585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b80dd-0816-46f6-a1ec-113602eab8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:42.417858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b80dd-0816-46f6-a1ec-113602eab8d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:42.430546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b80dd-0816-46f6-a1ec-113602eab8d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:42.432151 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b80dd-0816-46f6-a1ec-113602eab8d6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-3a7b80dd-0816-46f6-a1ec-113602eab8d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:42.448594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:42.449571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:42.450697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:42.450697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:42.450697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:42.450697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 247/470] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:42.450927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b80dd-0816-46f6-a1ec-113602eab8d6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:42.540423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2af094-605f-42b5-82d3-96a8fab220c0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:42.579065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a99c14-6d39-4d87-b7ec-36d7f0b0512a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:42.579678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a99c14-6d39-4d87-b7ec-36d7f0b0512a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:42.579959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a99c14-6d39-4d87-b7ec-36d7f0b0512a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:42.580236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a99c14-6d39-4d87-b7ec-36d7f0b0512a None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:42.580895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a99c14-6d39-4d87-b7ec-36d7f0b0512a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:42.581201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a99c14-6d39-4d87-b7ec-36d7f0b0512a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:42.581490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b80dd-0816-46f6-a1ec-113602eab8d6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:42.582615 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a99c14-6d39-4d87-b7ec-36d7f0b0512a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:42.599577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 224/471] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:42.599577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a99c14-6d39-4d87-b7ec-36d7f0b0512a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:42.624168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2486c0a7-a853-437a-8406-c2b85a5d5bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:42.624851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486c0a7-a853-437a-8406-c2b85a5d5bbf None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:42.625056 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486c0a7-a853-437a-8406-c2b85a5d5bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:42.625333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2486c0a7-a853-437a-8406-c2b85a5d5bbf None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:42.625987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486c0a7-a853-437a-8406-c2b85a5d5bbf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:42.627524 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2486c0a7-a853-437a-8406-c2b85a5d5bbf None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:42.632322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:42.648985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:42.667072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 248/472] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:08:42 2022] POST /identity/v3/users => generated 248 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:08:42.668542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c159e14-fe19-46f8-a8c3-b356157bb32c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:42.669116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c159e14-fe19-46f8-a8c3-b356157bb32c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:42.669116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c159e14-fe19-46f8-a8c3-b356157bb32c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:42.670447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 249/473] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:42.671184 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c159e14-fe19-46f8-a8c3-b356157bb32c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:42.721099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a99c14-6d39-4d87-b7ec-36d7f0b0512a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:42.767747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce574e-8871-4f90-8d39-d1a25227535f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:42.768634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce574e-8871-4f90-8d39-d1a25227535f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:42.769089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce574e-8871-4f90-8d39-d1a25227535f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:42.769499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce574e-8871-4f90-8d39-d1a25227535f None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:42.770339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce574e-8871-4f90-8d39-d1a25227535f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:42.770804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce574e-8871-4f90-8d39-d1a25227535f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:42.772444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce574e-8871-4f90-8d39-d1a25227535f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:42.793360 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 225/474] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:42.795094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfadd075-070a-4963-8e59-85b20a5b1f43 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:42.795548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfadd075-070a-4963-8e59-85b20a5b1f43 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:42.796166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfadd075-070a-4963-8e59-85b20a5b1f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:42.819475 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bfadd075-070a-4963-8e59-85b20a5b1f43 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:42.821212 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bfadd075-070a-4963-8e59-85b20a5b1f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:42.827079 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c159e14-fe19-46f8-a8c3-b356157bb32c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:42.881544 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e668e-deb4-4039-8328-499fbac60867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:42.883032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e668e-deb4-4039-8328-499fbac60867 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:42.883032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e668e-deb4-4039-8328-499fbac60867 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:42.883032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e668e-deb4-4039-8328-499fbac60867 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:42.885213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e668e-deb4-4039-8328-499fbac60867 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:42.886115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234e668e-deb4-4039-8328-499fbac60867 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:42.917533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 250/475] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:08:42 2022] POST /identity/v3/users => generated 248 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:08:42.918144 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e668e-deb4-4039-8328-499fbac60867 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:42.928359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 226/476] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:42.987395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:43.044095 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e668e-deb4-4039-8328-499fbac60867 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:43.100886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a76b9-fe2f-442c-96b7-0415404f2d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:43.101518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a76b9-fe2f-442c-96b7-0415404f2d42 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:43.101800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a76b9-fe2f-442c-96b7-0415404f2d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:43.102091 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a76b9-fe2f-442c-96b7-0415404f2d42 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:43.102763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a76b9-fe2f-442c-96b7-0415404f2d42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:43.104800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021a76b9-fe2f-442c-96b7-0415404f2d42 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:43.139376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 251/477] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:08:42 2022] POST /identity/v3/users => generated 249 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:08:43.275573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfadd075-070a-4963-8e59-85b20a5b1f43 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:43.310893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:43.407916 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfadd075-070a-4963-8e59-85b20a5b1f43 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:43.451915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fdcbc-7f1e-4f64-a07d-5b673fe5f919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:43.452533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fdcbc-7f1e-4f64-a07d-5b673fe5f919 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:43.452813 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fdcbc-7f1e-4f64-a07d-5b673fe5f919 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:43.453086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fdcbc-7f1e-4f64-a07d-5b673fe5f919 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:43.453778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fdcbc-7f1e-4f64-a07d-5b673fe5f919 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:43.455408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fdcbc-7f1e-4f64-a07d-5b673fe5f919 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:43.467095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.sql.core [None req-068fdcbc-7f1e-4f64-a07d-5b673fe5f919 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 21 16:08:43.467095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 21 16:08:43.467095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [parameters: {'id': 'bd576338e00941699caf2d94a07e5b1d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 21 16:08:43.467095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81425) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-068fdcbc-7f1e-4f64-a07d-5b673fe5f919 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application conn.query(q) Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result.read() Sep 21 16:08:43.498503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return ref.to_dict() Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 16:08:43.500107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application next(self.gen) Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application yield resource Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application next(self.gen) Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.session.rollback() Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.force_reraise() Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise self.value Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application session.commit() Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 21 16:08:43.501960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._prepare_impl() Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.session.flush() Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._flush(objects) Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application compat.raise_( Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application flush_context.execute() Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rec.execute(self) Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 21 16:08:43.503767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 16:08:43.507056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application conn.query(q) Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result.read() Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 16:08:43.510472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [parameters: {'id': 'bd576338e00941699caf2d94a07e5b1d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:43.514811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 21 16:08:43.515595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:43.516414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 227/478] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Sep 21 16:08:43 2022] POST /identity/v3/roles => generated 144 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:43.516479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a76b9-fe2f-442c-96b7-0415404f2d42 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:43.640583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a76b9-fe2f-442c-96b7-0415404f2d42 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:43.657149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:43.689319 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9db5944-0c55-4c07-b2cb-c7fd4476bf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:43.689967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db5944-0c55-4c07-b2cb-c7fd4476bf1d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:43.690364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db5944-0c55-4c07-b2cb-c7fd4476bf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:43.690648 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9db5944-0c55-4c07-b2cb-c7fd4476bf1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:43.705397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9db5944-0c55-4c07-b2cb-c7fd4476bf1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e9db5944-0c55-4c07-b2cb-c7fd4476bf1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:43.710918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:43.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:43.716030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:43.716030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:43.716030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:43.716030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:43.716030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:43.716030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:43.716030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:43.716030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:43.716030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:43.716030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:43.716030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 21 16:08:43.716030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:43.717563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 252/479] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:08:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:43.719644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fdcbc-7f1e-4f64-a07d-5b673fe5f919 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:43.948819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:43.980691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fdcbc-7f1e-4f64-a07d-5b673fe5f919 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:44.042113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5b49e-d9b7-49e6-8628-96af9383f24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:44.043199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5b49e-d9b7-49e6-8628-96af9383f24f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:44.043787 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5b49e-d9b7-49e6-8628-96af9383f24f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:44.044284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5b49e-d9b7-49e6-8628-96af9383f24f None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:44.045542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5b49e-d9b7-49e6-8628-96af9383f24f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:44.046109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5b49e-d9b7-49e6-8628-96af9383f24f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:44.048935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5b49e-d9b7-49e6-8628-96af9383f24f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:44.065601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 228/480] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:45.426247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b659c39-a18a-41dc-8f0e-9d6b0c59dc68 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:45.426767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b659c39-a18a-41dc-8f0e-9d6b0c59dc68 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:45.427194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b659c39-a18a-41dc-8f0e-9d6b0c59dc68 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:45.430068 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 253/481] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:08:45.437808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b153d59-e2a7-4f34-ac79-b08e4fe2845e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:45.438774 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b153d59-e2a7-4f34-ac79-b08e4fe2845e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:45.439252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b153d59-e2a7-4f34-ac79-b08e4fe2845e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:45.467145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2b153d59-e2a7-4f34-ac79-b08e4fe2845e None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:45.468742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2b153d59-e2a7-4f34-ac79-b08e4fe2845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:45.564431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 229/482] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:08:45.967266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b659c39-a18a-41dc-8f0e-9d6b0c59dc68 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:46.108230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b659c39-a18a-41dc-8f0e-9d6b0c59dc68 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:46.148040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6c119-6c2b-4440-b8b1-ef6c62802852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:46.148687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6c119-6c2b-4440-b8b1-ef6c62802852 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:46.148968 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6c119-6c2b-4440-b8b1-ef6c62802852 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:46.149256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6c119-6c2b-4440-b8b1-ef6c62802852 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:46.162348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6c119-6c2b-4440-b8b1-ef6c62802852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-d6f6c119-6c2b-4440-b8b1-ef6c62802852 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:46.167721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:46.168716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:46.169538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:46.169538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:46.169538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:46.169538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:46.169538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:46.169538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:46.169538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:46.169538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:46.169538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:46.169538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:46.169538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:46.169538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:46.171894 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 254/483] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:08:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:46.179419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b153d59-e2a7-4f34-ac79-b08e4fe2845e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:46.316106 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:46.355435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b153d59-e2a7-4f34-ac79-b08e4fe2845e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:46.409893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28fba31-1fb6-4f8f-bebd-f42ca0d169ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:46.410923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fba31-1fb6-4f8f-bebd-f42ca0d169ff None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:46.411405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fba31-1fb6-4f8f-bebd-f42ca0d169ff None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:46.411874 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fba31-1fb6-4f8f-bebd-f42ca0d169ff None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:46.412764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fba31-1fb6-4f8f-bebd-f42ca0d169ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:46.413293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fba31-1fb6-4f8f-bebd-f42ca0d169ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:46.414919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28fba31-1fb6-4f8f-bebd-f42ca0d169ff None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:46.425932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 230/484] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:08:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:46.433471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6c119-6c2b-4440-b8b1-ef6c62802852 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:46.443554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fdb12-f2e2-41e5-acd5-800eb64a2953 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:46.444132 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fdb12-f2e2-41e5-acd5-800eb64a2953 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:46.444383 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fdb12-f2e2-41e5-acd5-800eb64a2953 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:46.446862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 231/485] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:08:46.449161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5f16b-0d79-4a39-982f-d014ea76431b None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:46.449715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5f16b-0d79-4a39-982f-d014ea76431b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:46.450487 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5f16b-0d79-4a39-982f-d014ea76431b None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:46.452582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 232/486] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:46.461093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89357bf6-3b01-4559-b286-0b59263257e3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:46.461720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89357bf6-3b01-4559-b286-0b59263257e3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:46.462224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89357bf6-3b01-4559-b286-0b59263257e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:46.492876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:46.505145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-89357bf6-3b01-4559-b286-0b59263257e3 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:46.509679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-89357bf6-3b01-4559-b286-0b59263257e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:46.578679 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6c119-6c2b-4440-b8b1-ef6c62802852 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:46.615847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d9d8f-c0a8-4eb9-86a9-4ee6e6618126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:46.616495 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d9d8f-c0a8-4eb9-86a9-4ee6e6618126 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:46.616776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d9d8f-c0a8-4eb9-86a9-4ee6e6618126 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:46.617050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d9d8f-c0a8-4eb9-86a9-4ee6e6618126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:46.629314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d9d8f-c0a8-4eb9-86a9-4ee6e6618126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:46.630837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d9d8f-c0a8-4eb9-86a9-4ee6e6618126 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:46.646325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 233/487] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:46.646325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86428414-d060-4dd4-92e4-24a861305139 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:46.646325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86428414-d060-4dd4-92e4-24a861305139 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:46.646325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86428414-d060-4dd4-92e4-24a861305139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-bb6d9d8f-c0a8-4eb9-86a9-4ee6e6618126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:46.647050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:46.647928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:46.648788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:46.648788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:46.648788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:46.648788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 255/488] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:46.649846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31da7e-5e5c-48eb-a29f-13a32963a688 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:46.650155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31da7e-5e5c-48eb-a29f-13a32963a688 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:46.650433 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31da7e-5e5c-48eb-a29f-13a32963a688 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:46.654101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 256/489] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:46.657226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31da7e-5e5c-48eb-a29f-13a32963a688 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:46.674724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-86428414-d060-4dd4-92e4-24a861305139 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:46.676026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-86428414-d060-4dd4-92e4-24a861305139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:46.764699 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:46.778538 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 234/490] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:46.780014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2573bfe-6f52-4b15-b18a-bd1c6323f851 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:46.780307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2573bfe-6f52-4b15-b18a-bd1c6323f851 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:46.780590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2573bfe-6f52-4b15-b18a-bd1c6323f851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:46.796227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:46.819233 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f2573bfe-6f52-4b15-b18a-bd1c6323f851 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:46.820596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f2573bfe-6f52-4b15-b18a-bd1c6323f851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:46.831972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31da7e-5e5c-48eb-a29f-13a32963a688 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:46.899502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624f8c2-c6b9-4e6f-8a40-182e87ddcf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:46.900201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624f8c2-c6b9-4e6f-8a40-182e87ddcf27 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:46.900541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624f8c2-c6b9-4e6f-8a40-182e87ddcf27 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:46.900906 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624f8c2-c6b9-4e6f-8a40-182e87ddcf27 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:46.901642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624f8c2-c6b9-4e6f-8a40-182e87ddcf27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:46.901948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624f8c2-c6b9-4e6f-8a40-182e87ddcf27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:46.903391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624f8c2-c6b9-4e6f-8a40-182e87ddcf27 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:46.938294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 235/491] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:46.942459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 257/492] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:46.950806 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2573bfe-6f52-4b15-b18a-bd1c6323f851 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:47.064112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2573bfe-6f52-4b15-b18a-bd1c6323f851 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:47.096273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:47.109650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7488dbc-eead-4d5c-8f68-287eefdf75d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:47.110323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7488dbc-eead-4d5c-8f68-287eefdf75d4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:47.110603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7488dbc-eead-4d5c-8f68-287eefdf75d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:47.110875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7488dbc-eead-4d5c-8f68-287eefdf75d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:47.119010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7488dbc-eead-4d5c-8f68-287eefdf75d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:47.121083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7488dbc-eead-4d5c-8f68-287eefdf75d4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:47.131670 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624f8c2-c6b9-4e6f-8a40-182e87ddcf27 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c7488dbc-eead-4d5c-8f68-287eefdf75d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:08:47.144620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:47.145658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:47.146663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:47.146663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:47.146663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:08:47.146663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:08:47.146663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:08:47.146663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:08:47.146663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:08:47.146663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:08:47.146663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:08:47.146663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:47.150761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 236/493] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Sep 21 16:08:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:47.156310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7488dbc-eead-4d5c-8f68-287eefdf75d4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:47.271368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624f8c2-c6b9-4e6f-8a40-182e87ddcf27 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:47.293551 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7488dbc-eead-4d5c-8f68-287eefdf75d4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:47.313698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3baf14-8056-4d62-aa3b-a1977eea94cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:47.314377 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3baf14-8056-4d62-aa3b-a1977eea94cf None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:47.314721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3baf14-8056-4d62-aa3b-a1977eea94cf None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:47.314979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3baf14-8056-4d62-aa3b-a1977eea94cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:47.327710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3baf14-8056-4d62-aa3b-a1977eea94cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:47.331534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba22bf-5f69-47b1-88cb-69158f96aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:47.332143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba22bf-5f69-47b1-88cb-69158f96aa4d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:47.332418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba22bf-5f69-47b1-88cb-69158f96aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:47.332694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba22bf-5f69-47b1-88cb-69158f96aa4d None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-cf3baf14-8056-4d62-aa3b-a1977eea94cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:47.332929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:47.336197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:47.340297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:47.340297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:47.340297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:47.340297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:47.340297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:47.340297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:47.340297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:47.340297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:47.340297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:47.340297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:47.340297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:47.340297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:47.342610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba22bf-5f69-47b1-88cb-69158f96aa4d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:47.342610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba22bf-5f69-47b1-88cb-69158f96aa4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:47.342610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fba22bf-5f69-47b1-88cb-69158f96aa4d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:47.343233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 258/494] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:08:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:08:47.343233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3baf14-8056-4d62-aa3b-a1977eea94cf None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:47.346089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:47.379258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 237/495] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Sep 21 16:08:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:47.380036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba22bf-5f69-47b1-88cb-69158f96aa4d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:47.432416 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:47.458715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3baf14-8056-4d62-aa3b-a1977eea94cf None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:47.491071 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fba22bf-5f69-47b1-88cb-69158f96aa4d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:47.504455 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971ba21-603b-49a9-9419-38c043b0499e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:47.505338 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971ba21-603b-49a9-9419-38c043b0499e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:47.505744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971ba21-603b-49a9-9419-38c043b0499e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:47.506186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971ba21-603b-49a9-9419-38c043b0499e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:47.520427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971ba21-603b-49a9-9419-38c043b0499e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-d971ba21-603b-49a9-9419-38c043b0499e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:47.525798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:47.527065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 259/496] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:08:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:47.527978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971ba21-603b-49a9-9419-38c043b0499e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:47.534552 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce29134-ad06-4c6a-9e35-1109c79b903c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:47.535305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce29134-ad06-4c6a-9e35-1109c79b903c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:47.535673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce29134-ad06-4c6a-9e35-1109c79b903c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:47.536033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce29134-ad06-4c6a-9e35-1109c79b903c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:47.548548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce29134-ad06-4c6a-9e35-1109c79b903c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-3ce29134-ad06-4c6a-9e35-1109c79b903c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:47.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:47.555129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:47.555990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 238/497] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:08:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:47.556681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207c928-772c-49dc-9b8f-0dfcc66d0f4f None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:47.557272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207c928-772c-49dc-9b8f-0dfcc66d0f4f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:47.557652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207c928-772c-49dc-9b8f-0dfcc66d0f4f None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:47.558882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 239/498] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:47.559617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207c928-772c-49dc-9b8f-0dfcc66d0f4f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:47.607182 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:47.657363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971ba21-603b-49a9-9419-38c043b0499e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:47.691448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207c928-772c-49dc-9b8f-0dfcc66d0f4f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:47.703643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:47.706636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724213d-5f00-44aa-a63d-4531f2beb4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:47.707578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724213d-5f00-44aa-a63d-4531f2beb4ed None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:47.707912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724213d-5f00-44aa-a63d-4531f2beb4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:47.708228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724213d-5f00-44aa-a63d-4531f2beb4ed None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:47.709037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724213d-5f00-44aa-a63d-4531f2beb4ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:47.709427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724213d-5f00-44aa-a63d-4531f2beb4ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:47.711375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724213d-5f00-44aa-a63d-4531f2beb4ed None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:47.722468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 260/499] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:08:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:47.723914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724213d-5f00-44aa-a63d-4531f2beb4ed None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:47.749813 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ecb05-ad90-4cd4-b0be-eb03c0a873ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:47.750836 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ecb05-ad90-4cd4-b0be-eb03c0a873ba None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:47.751282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ecb05-ad90-4cd4-b0be-eb03c0a873ba None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:47.751721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ecb05-ad90-4cd4-b0be-eb03c0a873ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:47.832469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ecb05-ad90-4cd4-b0be-eb03c0a873ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:47.833948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ecb05-ad90-4cd4-b0be-eb03c0a873ba None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:47.848630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724213d-5f00-44aa-a63d-4531f2beb4ed None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:47.877969 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:47.891633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14978b-db81-4c91-8b3c-497270eed94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:47.892334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14978b-db81-4c91-8b3c-497270eed94f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:47.892684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14978b-db81-4c91-8b3c-497270eed94f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:47.893071 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14978b-db81-4c91-8b3c-497270eed94f None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:47.893862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14978b-db81-4c91-8b3c-497270eed94f None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:47.894316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14978b-db81-4c91-8b3c-497270eed94f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:47.895896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa14978b-db81-4c91-8b3c-497270eed94f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:47.905673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 261/500] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:08:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:47.906569 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14978b-db81-4c91-8b3c-497270eed94f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:47.948401 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 240/501] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:47.949568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff62ba-8477-4005-9e66-be23be31f556 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:47.950129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff62ba-8477-4005-9e66-be23be31f556 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:47.950403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff62ba-8477-4005-9e66-be23be31f556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:47.969296 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-26ff62ba-8477-4005-9e66-be23be31f556 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:47.970570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-26ff62ba-8477-4005-9e66-be23be31f556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:47.972820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:48.016955 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa14978b-db81-4c91-8b3c-497270eed94f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:48.051570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 241/502] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:48.053054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff62ba-8477-4005-9e66-be23be31f556 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:48.057847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef310fbc-d0e7-43fd-8a5c-74970f565361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:48.058599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef310fbc-d0e7-43fd-8a5c-74970f565361 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:48.059061 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef310fbc-d0e7-43fd-8a5c-74970f565361 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:48.059506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef310fbc-d0e7-43fd-8a5c-74970f565361 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:48.060612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef310fbc-d0e7-43fd-8a5c-74970f565361 None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:48.061107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef310fbc-d0e7-43fd-8a5c-74970f565361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:48.063568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef310fbc-d0e7-43fd-8a5c-74970f565361 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:48.087893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 262/503] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:08:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:48.087893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef310fbc-d0e7-43fd-8a5c-74970f565361 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:48.168234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:48.207511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff62ba-8477-4005-9e66-be23be31f556 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:48.249763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef310fbc-d0e7-43fd-8a5c-74970f565361 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:48.252831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb77407-fdfb-4481-a2e8-9216cd2a25ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:48.253490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb77407-fdfb-4481-a2e8-9216cd2a25ce None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:48.253773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb77407-fdfb-4481-a2e8-9216cd2a25ce None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:48.254129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb77407-fdfb-4481-a2e8-9216cd2a25ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:48.262016 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:48.267647 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb77407-fdfb-4481-a2e8-9216cd2a25ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:48.269132 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb77407-fdfb-4481-a2e8-9216cd2a25ce None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1bb77407-fdfb-4481-a2e8-9216cd2a25ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:48.286412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:48.287466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:48.288499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:48.288499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:48.288499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:48.289947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 242/504] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:48.290948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb77407-fdfb-4481-a2e8-9216cd2a25ce None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:48.307377 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bf0db-b5f2-451a-8059-8891c5bbdcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:48.308278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bf0db-b5f2-451a-8059-8891c5bbdcf3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:48.308657 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bf0db-b5f2-451a-8059-8891c5bbdcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:48.308975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bf0db-b5f2-451a-8059-8891c5bbdcf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:48.322471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bf0db-b5f2-451a-8059-8891c5bbdcf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:48.324443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bf0db-b5f2-451a-8059-8891c5bbdcf3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4b4bf0db-b5f2-451a-8059-8891c5bbdcf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:48.343083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:48.344404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:48.345952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:48.345952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:48.345952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:48.345952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 263/505] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:48.345952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bf0db-b5f2-451a-8059-8891c5bbdcf3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:48.408289 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb77407-fdfb-4481-a2e8-9216cd2a25ce None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:48.449142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0dc3a7-61c4-4525-b50b-3947f0360b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:48.450418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0dc3a7-61c4-4525-b50b-3947f0360b96 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:48.450761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0dc3a7-61c4-4525-b50b-3947f0360b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:48.451157 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0dc3a7-61c4-4525-b50b-3947f0360b96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:48.465754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:48.469505 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0dc3a7-61c4-4525-b50b-3947f0360b96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:48.470694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bf0db-b5f2-451a-8059-8891c5bbdcf3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:48.472054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0dc3a7-61c4-4525-b50b-3947f0360b96 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-9e0dc3a7-61c4-4525-b50b-3947f0360b96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:48.495264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:48.496780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:48.497626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:48.497626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:48.497626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:48.497626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 243/506] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:48.497849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0dc3a7-61c4-4525-b50b-3947f0360b96 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:48.519334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac58684-4bed-492b-8e05-ba48be96ff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:48.519979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac58684-4bed-492b-8e05-ba48be96ff30 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:48.520256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac58684-4bed-492b-8e05-ba48be96ff30 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:48.520530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac58684-4bed-492b-8e05-ba48be96ff30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:48.533038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac58684-4bed-492b-8e05-ba48be96ff30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:48.534514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac58684-4bed-492b-8e05-ba48be96ff30 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6ac58684-4bed-492b-8e05-ba48be96ff30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:48.549421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:48.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:48.552414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:48.552414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:48.552414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:48.552414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 264/507] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:48.552627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:48.552696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac58684-4bed-492b-8e05-ba48be96ff30 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:48.627414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0dc3a7-61c4-4525-b50b-3947f0360b96 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:48.663024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4543175-8211-4dfd-ac2e-da4bb60db56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:48.663922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4543175-8211-4dfd-ac2e-da4bb60db56d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:48.664204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4543175-8211-4dfd-ac2e-da4bb60db56d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:48.664475 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4543175-8211-4dfd-ac2e-da4bb60db56d None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:48.665124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4543175-8211-4dfd-ac2e-da4bb60db56d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:48.665423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4543175-8211-4dfd-ac2e-da4bb60db56d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:48.666979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4543175-8211-4dfd-ac2e-da4bb60db56d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:48.671478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac58684-4bed-492b-8e05-ba48be96ff30 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:48.679366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 244/508] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:48.680203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4543175-8211-4dfd-ac2e-da4bb60db56d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:48.712537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae32a0c-fe3c-406a-8a04-1731c46e07ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:48.713836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae32a0c-fe3c-406a-8a04-1731c46e07ab None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:48.714156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae32a0c-fe3c-406a-8a04-1731c46e07ab None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:48.714425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae32a0c-fe3c-406a-8a04-1731c46e07ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:48.725484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae32a0c-fe3c-406a-8a04-1731c46e07ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:48.726897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae32a0c-fe3c-406a-8a04-1731c46e07ab None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4ae32a0c-fe3c-406a-8a04-1731c46e07ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:48.742490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:48.744853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:48.746677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:48.746677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:48.746677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:48.746677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 265/509] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:48.746677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae32a0c-fe3c-406a-8a04-1731c46e07ab None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:48.749804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:48.798876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4543175-8211-4dfd-ac2e-da4bb60db56d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:48.800512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:48.859755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac6d73-507c-45b1-9e48-c543494900c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:48.860577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac6d73-507c-45b1-9e48-c543494900c4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:48.861027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac6d73-507c-45b1-9e48-c543494900c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:48.861478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac6d73-507c-45b1-9e48-c543494900c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:48.862604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac6d73-507c-45b1-9e48-c543494900c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:48.863100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac6d73-507c-45b1-9e48-c543494900c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:48.865394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac6d73-507c-45b1-9e48-c543494900c4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:48.878468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae32a0c-fe3c-406a-8a04-1731c46e07ab None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:48.884434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 245/510] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:48.886368 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac6d73-507c-45b1-9e48-c543494900c4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:48.921582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160427c-fa17-4891-82c0-2b1ecf16d4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:48.922419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160427c-fa17-4891-82c0-2b1ecf16d4af None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:48.922730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160427c-fa17-4891-82c0-2b1ecf16d4af None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:48.923101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160427c-fa17-4891-82c0-2b1ecf16d4af None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:48.923933 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160427c-fa17-4891-82c0-2b1ecf16d4af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:48.924333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160427c-fa17-4891-82c0-2b1ecf16d4af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:48.925986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160427c-fa17-4891-82c0-2b1ecf16d4af None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:48.939722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 266/511] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:48.941393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160427c-fa17-4891-82c0-2b1ecf16d4af None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:49.000919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac6d73-507c-45b1-9e48-c543494900c4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:49.026358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:49.036448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370874a-f4e2-4ac8-b382-b30556ecfb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:49.037057 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370874a-f4e2-4ac8-b382-b30556ecfb0e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:49.037332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370874a-f4e2-4ac8-b382-b30556ecfb0e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:49.037597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370874a-f4e2-4ac8-b382-b30556ecfb0e None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:49.038272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370874a-f4e2-4ac8-b382-b30556ecfb0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:49.038573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370874a-f4e2-4ac8-b382-b30556ecfb0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:49.039931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370874a-f4e2-4ac8-b382-b30556ecfb0e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:49.053124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 246/512] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:49.054020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370874a-f4e2-4ac8-b382-b30556ecfb0e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:49.077548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160427c-fa17-4891-82c0-2b1ecf16d4af None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:49.080833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:49.117863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec813d8f-cfd5-44b0-8306-a7f79f127b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:49.118715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec813d8f-cfd5-44b0-8306-a7f79f127b5d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:49.119060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec813d8f-cfd5-44b0-8306-a7f79f127b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:49.119473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec813d8f-cfd5-44b0-8306-a7f79f127b5d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:49.128515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec813d8f-cfd5-44b0-8306-a7f79f127b5d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:49.130260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec813d8f-cfd5-44b0-8306-a7f79f127b5d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ec813d8f-cfd5-44b0-8306-a7f79f127b5d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:08:49.144646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:49.146464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:49.148458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:49.148458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:49.148458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:08:49.148458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:08:49.148458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:08:49.148458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:08:49.148458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:08:49.148458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:08:49.148458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 21 16:08:49.148458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:49.148458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 267/513] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:49.148458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec813d8f-cfd5-44b0-8306-a7f79f127b5d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:49.170913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370874a-f4e2-4ac8-b382-b30556ecfb0e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:49.208143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f3442-b13c-4221-8695-399751d311d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:49.208746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f3442-b13c-4221-8695-399751d311d7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:49.209021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f3442-b13c-4221-8695-399751d311d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:49.209296 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f3442-b13c-4221-8695-399751d311d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:49.209936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f3442-b13c-4221-8695-399751d311d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:49.210275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f3442-b13c-4221-8695-399751d311d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:49.211638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f3442-b13c-4221-8695-399751d311d7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:49.228839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 247/514] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:49.228839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f3442-b13c-4221-8695-399751d311d7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:49.261391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec813d8f-cfd5-44b0-8306-a7f79f127b5d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:49.287783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:49.300517 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97928bb2-7a80-4024-a7a0-a4e03a6fab1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:49.301412 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97928bb2-7a80-4024-a7a0-a4e03a6fab1d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:49.301735 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97928bb2-7a80-4024-a7a0-a4e03a6fab1d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:49.302176 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97928bb2-7a80-4024-a7a0-a4e03a6fab1d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:49.314727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97928bb2-7a80-4024-a7a0-a4e03a6fab1d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:49.317140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97928bb2-7a80-4024-a7a0-a4e03a6fab1d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:49.320324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-97928bb2-7a80-4024-a7a0-a4e03a6fab1d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:08:49.337046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:49.338834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:49.339986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:49.339986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:49.339986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:08:49.339986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:08:49.339986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:08:49.339986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:08:49.339986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:08:49.339986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:08:49.339986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 16:08:49.339986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:49.339986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 268/515] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:08:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:49.341101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97928bb2-7a80-4024-a7a0-a4e03a6fab1d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:49.350680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f3442-b13c-4221-8695-399751d311d7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:49.388397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae06f68-7cff-4d51-bb13-0a0379a8b1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:49.389176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae06f68-7cff-4d51-bb13-0a0379a8b1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:49.389455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae06f68-7cff-4d51-bb13-0a0379a8b1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:49.389724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae06f68-7cff-4d51-bb13-0a0379a8b1d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:49.397134 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae06f68-7cff-4d51-bb13-0a0379a8b1d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:49.398610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae06f68-7cff-4d51-bb13-0a0379a8b1d4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-eae06f68-7cff-4d51-bb13-0a0379a8b1d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:08:49.411569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:49.412667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:49.413629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:08:49.413629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:08:49.413629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:08:49.413629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:08:49.413629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:08:49.413629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:08:49.413629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:08:49.413629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:49.413629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 248/516] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:49.413629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae06f68-7cff-4d51-bb13-0a0379a8b1d4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:49.469934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97928bb2-7a80-4024-a7a0-a4e03a6fab1d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:49.507881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a999e2a-017b-4edf-b853-cd3f3f82c3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:49.508915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a999e2a-017b-4edf-b853-cd3f3f82c3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:49.509196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a999e2a-017b-4edf-b853-cd3f3f82c3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:49.509466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a999e2a-017b-4edf-b853-cd3f3f82c3e1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:49.519343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a999e2a-017b-4edf-b853-cd3f3f82c3e1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:49.520725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a999e2a-017b-4edf-b853-cd3f3f82c3e1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:49.532507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae06f68-7cff-4d51-bb13-0a0379a8b1d4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-7a999e2a-017b-4edf-b853-cd3f3f82c3e1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:08:49.533060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:49.534746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:49.536489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:49.536489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:49.536489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:08:49.536489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:08:49.536489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:08:49.536489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:08:49.536489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:08:49.536489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:08:49.536489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 16:08:49.536489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:49.536489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 269/517] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:49.536489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a999e2a-017b-4edf-b853-cd3f3f82c3e1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:49.574624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52b4a9-981f-4164-96e1-345e3dc30d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:49.575729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52b4a9-981f-4164-96e1-345e3dc30d85 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:49.575977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52b4a9-981f-4164-96e1-345e3dc30d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:49.576305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52b4a9-981f-4164-96e1-345e3dc30d85 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:49.577192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52b4a9-981f-4164-96e1-345e3dc30d85 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:49.577640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52b4a9-981f-4164-96e1-345e3dc30d85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:49.579300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52b4a9-981f-4164-96e1-345e3dc30d85 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:49.587923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:49.594591 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:49.599270 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 249/518] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Sep 21 16:08:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:49.600916 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52b4a9-981f-4164-96e1-345e3dc30d85 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:49.667977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a999e2a-017b-4edf-b853-cd3f3f82c3e1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:49.710874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4485e3-5857-4b76-9a7c-2e49024647af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:49.711559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4485e3-5857-4b76-9a7c-2e49024647af None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:49.711902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4485e3-5857-4b76-9a7c-2e49024647af None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:49.712267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4485e3-5857-4b76-9a7c-2e49024647af None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:49.713554 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4485e3-5857-4b76-9a7c-2e49024647af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:49.715050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4485e3-5857-4b76-9a7c-2e49024647af None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:49.726074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52b4a9-981f-4164-96e1-345e3dc30d85 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:49.739358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 270/519] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:08:49 2022] POST /identity/v3/users => generated 247 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:08:49.740410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4485e3-5857-4b76-9a7c-2e49024647af None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:49.770403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6573bdd-a370-484e-94ea-c3a14fa3b662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:49.770953 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573bdd-a370-484e-94ea-c3a14fa3b662 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:49.771212 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573bdd-a370-484e-94ea-c3a14fa3b662 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:49.771466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6573bdd-a370-484e-94ea-c3a14fa3b662 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:49.772251 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6573bdd-a370-484e-94ea-c3a14fa3b662 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:49.772556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6573bdd-a370-484e-94ea-c3a14fa3b662 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:49.774084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6573bdd-a370-484e-94ea-c3a14fa3b662 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:49.796150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 250/520] 10.222.0.64 () {56 vars in 1139 bytes} [Wed Sep 21 16:08:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:49.797152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6573bdd-a370-484e-94ea-c3a14fa3b662 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:49.884364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:49.900400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:49.903939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4485e3-5857-4b76-9a7c-2e49024647af None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:49.946248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6573bdd-a370-484e-94ea-c3a14fa3b662 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:49.956268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382b308-a957-48cb-83a3-0af8f37b680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:49.956893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382b308-a957-48cb-83a3-0af8f37b680a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:49.957168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382b308-a957-48cb-83a3-0af8f37b680a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:49.957435 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382b308-a957-48cb-83a3-0af8f37b680a None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:49.958797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382b308-a957-48cb-83a3-0af8f37b680a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:49.958797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382b308-a957-48cb-83a3-0af8f37b680a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:49.960003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382b308-a957-48cb-83a3-0af8f37b680a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:49.983355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 271/521] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:08:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:49.984245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382b308-a957-48cb-83a3-0af8f37b680a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:50.012391 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525daac-07f8-48b2-933f-05af4cf0a24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:50.013019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525daac-07f8-48b2-933f-05af4cf0a24c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:50.013291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525daac-07f8-48b2-933f-05af4cf0a24c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:50.013563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525daac-07f8-48b2-933f-05af4cf0a24c None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:50.014346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525daac-07f8-48b2-933f-05af4cf0a24c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:50.014653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525daac-07f8-48b2-933f-05af4cf0a24c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:50.016127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d525daac-07f8-48b2-933f-05af4cf0a24c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:50.043500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 251/522] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Sep 21 16:08:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:50.043500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525daac-07f8-48b2-933f-05af4cf0a24c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:50.130568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382b308-a957-48cb-83a3-0af8f37b680a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:50.164006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525daac-07f8-48b2-933f-05af4cf0a24c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:50.183811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:50.193377 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182db785-1042-42d7-bfe0-9eed8957027e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:50.194438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182db785-1042-42d7-bfe0-9eed8957027e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:50.194941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182db785-1042-42d7-bfe0-9eed8957027e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:50.195444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182db785-1042-42d7-bfe0-9eed8957027e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:50.204155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6997b-9439-4701-bf92-43cc16f3e17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:50.204777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6997b-9439-4701-bf92-43cc16f3e17c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:50.205054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6997b-9439-4701-bf92-43cc16f3e17c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:50.205323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6997b-9439-4701-bf92-43cc16f3e17c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:50.249708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:50.284805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc6997b-9439-4701-bf92-43cc16f3e17c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:50.286724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc6997b-9439-4701-bf92-43cc16f3e17c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:50.325484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182db785-1042-42d7-bfe0-9eed8957027e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:50.328309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182db785-1042-42d7-bfe0-9eed8957027e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:50.421790 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 252/523] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:50.421790 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6997b-9439-4701-bf92-43cc16f3e17c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:50.449861 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:50.464208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 272/524] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:50.464208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182db785-1042-42d7-bfe0-9eed8957027e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:50.560208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:50.597391 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6997b-9439-4701-bf92-43cc16f3e17c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:50.622919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182db785-1042-42d7-bfe0-9eed8957027e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:50.656226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675b3fd-2115-4317-91b8-fe1da9ce13aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:50.657179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675b3fd-2115-4317-91b8-fe1da9ce13aa None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:50.658260 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675b3fd-2115-4317-91b8-fe1da9ce13aa None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:50.658706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675b3fd-2115-4317-91b8-fe1da9ce13aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:50.659942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675b3fd-2115-4317-91b8-fe1da9ce13aa None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=f0acf23d224e4b16ad03cdd9bf7cae8e) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:50.660439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675b3fd-2115-4317-91b8-fe1da9ce13aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:50.662946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675b3fd-2115-4317-91b8-fe1da9ce13aa None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:50.672627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 253/525] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:08:50 2022] GET /identity/v3/role_assignments?user.id=f0acf23d224e4b16ad03cdd9bf7cae8e&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:50.673752 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675b3fd-2115-4317-91b8-fe1da9ce13aa None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:50.693990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc9c78-fb56-4af7-83d0-be58d959cc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:50.695525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc9c78-fb56-4af7-83d0-be58d959cc6c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:50.696200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc9c78-fb56-4af7-83d0-be58d959cc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:50.696833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc9c78-fb56-4af7-83d0-be58d959cc6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:50.824267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc9c78-fb56-4af7-83d0-be58d959cc6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:50.825680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:50.828362 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc9c78-fb56-4af7-83d0-be58d959cc6c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:50.871180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675b3fd-2115-4317-91b8-fe1da9ce13aa None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:50.936443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:50.944652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befebd0-0ac0-4c0e-9c9a-3656a6be9d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:50.946677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befebd0-0ac0-4c0e-9c9a-3656a6be9d94 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:50.947438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befebd0-0ac0-4c0e-9c9a-3656a6be9d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:50.948151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5befebd0-0ac0-4c0e-9c9a-3656a6be9d94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:50.975150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befebd0-0ac0-4c0e-9c9a-3656a6be9d94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:50.978352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5befebd0-0ac0-4c0e-9c9a-3656a6be9d94 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-5befebd0-0ac0-4c0e-9c9a-3656a6be9d94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:51.012298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:51.014019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:51.015762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:51.015762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:51.015762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:51.018432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 273/526] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:51.019318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 254/527] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:51.021569 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc9c78-fb56-4af7-83d0-be58d959cc6c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:51.036410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befebd0-0ac0-4c0e-9c9a-3656a6be9d94 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:51.215904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5befebd0-0ac0-4c0e-9c9a-3656a6be9d94 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:51.250343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc9c78-fb56-4af7-83d0-be58d959cc6c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:51.279661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:51.292317 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af2d2e-3321-426f-a2b1-11206eda094c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:51.293413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af2d2e-3321-426f-a2b1-11206eda094c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:51.293801 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af2d2e-3321-426f-a2b1-11206eda094c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:51.294252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7af2d2e-3321-426f-a2b1-11206eda094c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:51.319607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af2d2e-3321-426f-a2b1-11206eda094c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:51.322906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7af2d2e-3321-426f-a2b1-11206eda094c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:51.323923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5879ce4-cc01-4bc0-8b50-2f22f17e482d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:51.325469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5879ce4-cc01-4bc0-8b50-2f22f17e482d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:51.326165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5879ce4-cc01-4bc0-8b50-2f22f17e482d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:51.326777 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5879ce4-cc01-4bc0-8b50-2f22f17e482d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:51.347324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5879ce4-cc01-4bc0-8b50-2f22f17e482d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:51.350366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5879ce4-cc01-4bc0-8b50-2f22f17e482d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c7af2d2e-3321-426f-a2b1-11206eda094c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:51.354892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:51.355934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:51.356879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:51.356879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:51.356879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:51.362094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 255/528] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:51.362094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af2d2e-3321-426f-a2b1-11206eda094c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a5879ce4-cc01-4bc0-8b50-2f22f17e482d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:51.391271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:51.392324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:51.393264 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:51.393264 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:51.393264 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:51.395765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 274/529] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:51.397379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5879ce4-cc01-4bc0-8b50-2f22f17e482d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:51.443495 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:51.571410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7af2d2e-3321-426f-a2b1-11206eda094c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:51.615203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf57328-29ea-4903-8bdc-edb465f46d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:51.616487 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf57328-29ea-4903-8bdc-edb465f46d1f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:51.616910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf57328-29ea-4903-8bdc-edb465f46d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:51.617321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf57328-29ea-4903-8bdc-edb465f46d1f None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:51.618484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf57328-29ea-4903-8bdc-edb465f46d1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:51.618959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf57328-29ea-4903-8bdc-edb465f46d1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:51.621173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf57328-29ea-4903-8bdc-edb465f46d1f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:51.625009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5879ce4-cc01-4bc0-8b50-2f22f17e482d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:51.641165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 256/530] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:51.642403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf57328-29ea-4903-8bdc-edb465f46d1f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:51.691479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4193a53-0a50-4755-a95d-39d0d8fda8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:51.692684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4193a53-0a50-4755-a95d-39d0d8fda8fe None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:51.693300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4193a53-0a50-4755-a95d-39d0d8fda8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:51.694079 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4193a53-0a50-4755-a95d-39d0d8fda8fe None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:51.695652 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4193a53-0a50-4755-a95d-39d0d8fda8fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:51.696124 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4193a53-0a50-4755-a95d-39d0d8fda8fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:51.698819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:51.701142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4193a53-0a50-4755-a95d-39d0d8fda8fe None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:51.734651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 275/531] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:51.736490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4193a53-0a50-4755-a95d-39d0d8fda8fe None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:51.765865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf57328-29ea-4903-8bdc-edb465f46d1f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:51.807805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9b8cc-29b6-4c96-9b32-5789b36eb352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:51.808668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9b8cc-29b6-4c96-9b32-5789b36eb352 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:51.810167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9b8cc-29b6-4c96-9b32-5789b36eb352 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:51.810599 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad9b8cc-29b6-4c96-9b32-5789b36eb352 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:51.811430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9b8cc-29b6-4c96-9b32-5789b36eb352 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:51.811877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9b8cc-29b6-4c96-9b32-5789b36eb352 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:51.813383 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad9b8cc-29b6-4c96-9b32-5789b36eb352 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:51.832468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 257/532] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:51.832468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9b8cc-29b6-4c96-9b32-5789b36eb352 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:51.868637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:51.927384 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4193a53-0a50-4755-a95d-39d0d8fda8fe None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:51.971060 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad9b8cc-29b6-4c96-9b32-5789b36eb352 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:51.975145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:51.992897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22349535-c582-458a-a031-f34688fcd359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:51.994103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22349535-c582-458a-a031-f34688fcd359 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:51.994603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22349535-c582-458a-a031-f34688fcd359 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:51.995090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22349535-c582-458a-a031-f34688fcd359 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:52.009088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22349535-c582-458a-a031-f34688fcd359 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:52.012283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22349535-c582-458a-a031-f34688fcd359 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:52.017352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c20605-b2e6-43fb-858f-32584d77b0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:52.018782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c20605-b2e6-43fb-858f-32584d77b0ed None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:52.019301 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c20605-b2e6-43fb-858f-32584d77b0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:52.019768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c20605-b2e6-43fb-858f-32584d77b0ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:52.028428 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c20605-b2e6-43fb-858f-32584d77b0ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:52.030275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c20605-b2e6-43fb-858f-32584d77b0ed None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-22349535-c582-458a-a031-f34688fcd359 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:08:52.036720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:52.037593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:52.038462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:08:52.038462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:08:52.038462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:08:52.038462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:08:52.038462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:08:52.038462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:08:52.038462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:08:52.038462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:52.042618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 276/533] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:52.044221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22349535-c582-458a-a031-f34688fcd359 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-38c20605-b2e6-43fb-858f-32584d77b0ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:08:52.047482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:52.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:52.049244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:08:52.049244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:08:52.049244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:08:52.049244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:08:52.049244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:08:52.049244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:08:52.049244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:08:52.049244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:52.052181 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 258/534] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:52.053374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c20605-b2e6-43fb-858f-32584d77b0ed None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:52.221218 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22349535-c582-458a-a031-f34688fcd359 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:52.223332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c20605-b2e6-43fb-858f-32584d77b0ed None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:52.254028 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:52.263113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319d4c2-9394-41a6-931f-552844b046fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:52.264175 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319d4c2-9394-41a6-931f-552844b046fd None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:52.264689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319d4c2-9394-41a6-931f-552844b046fd None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:52.265179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319d4c2-9394-41a6-931f-552844b046fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:52.276109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f31caf-8725-41ea-9cfe-f5a01a224c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:52.276650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f31caf-8725-41ea-9cfe-f5a01a224c8c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:52.276930 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f31caf-8725-41ea-9cfe-f5a01a224c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:52.277200 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f31caf-8725-41ea-9cfe-f5a01a224c8c None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:52.277581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319d4c2-9394-41a6-931f-552844b046fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:52.277912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f31caf-8725-41ea-9cfe-f5a01a224c8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:52.278251 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f31caf-8725-41ea-9cfe-f5a01a224c8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:52.279575 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319d4c2-9394-41a6-931f-552844b046fd None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:52.279729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f31caf-8725-41ea-9cfe-f5a01a224c8c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-c319d4c2-9394-41a6-931f-552844b046fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:08:52.293590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:52.294588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:52.295422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:08:52.295422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:08:52.295422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:08:52.295422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:08:52.295422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:08:52.295422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:08:52.295422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:08:52.295422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:52.301398 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 277/535] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:52.301398 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319d4c2-9394-41a6-931f-552844b046fd None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:52.301643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 259/536] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:08:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:52.307570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639a927-b39e-4c64-b61b-83f7f25aac26 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:52.308272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639a927-b39e-4c64-b61b-83f7f25aac26 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:52.308550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639a927-b39e-4c64-b61b-83f7f25aac26 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:52.310202 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 260/537] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:52 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:52.310783 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639a927-b39e-4c64-b61b-83f7f25aac26 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:52.354081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:52.442674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319d4c2-9394-41a6-931f-552844b046fd None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:52.486142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abbc98-72c3-46e7-8cc4-bb2e00f6a9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:52.487055 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abbc98-72c3-46e7-8cc4-bb2e00f6a9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:52.487489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abbc98-72c3-46e7-8cc4-bb2e00f6a9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:52.487911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7abbc98-72c3-46e7-8cc4-bb2e00f6a9c4 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:52.489204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abbc98-72c3-46e7-8cc4-bb2e00f6a9c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:52.489832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abbc98-72c3-46e7-8cc4-bb2e00f6a9c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:52.492341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7abbc98-72c3-46e7-8cc4-bb2e00f6a9c4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:52.495580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639a927-b39e-4c64-b61b-83f7f25aac26 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:52.529822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 278/538] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:08:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:52.529822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abbc98-72c3-46e7-8cc4-bb2e00f6a9c4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:52.550687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a0a29-f0e2-49f7-9d86-ee06c8f6c477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:52.551558 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a0a29-f0e2-49f7-9d86-ee06c8f6c477 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:52.551981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a0a29-f0e2-49f7-9d86-ee06c8f6c477 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:52.552395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a0a29-f0e2-49f7-9d86-ee06c8f6c477 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:52.553284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a0a29-f0e2-49f7-9d86-ee06c8f6c477 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:52.553935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a0a29-f0e2-49f7-9d86-ee06c8f6c477 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:52.556209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a0a29-f0e2-49f7-9d86-ee06c8f6c477 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:52.572017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 261/539] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:08:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:52.575348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e166d31-d0a4-4f39-a341-57f47af19c66 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:52.575845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e166d31-d0a4-4f39-a341-57f47af19c66 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:52.576443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e166d31-d0a4-4f39-a341-57f47af19c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:52.601562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2e166d31-d0a4-4f39-a341-57f47af19c66 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:52.603502 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2e166d31-d0a4-4f39-a341-57f47af19c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:52.619589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:52.687140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7abbc98-72c3-46e7-8cc4-bb2e00f6a9c4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:52.714660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 262/540] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:52.715909 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e166d31-d0a4-4f39-a341-57f47af19c66 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:52.743019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:52.744214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f1123-1a72-45b9-8eb5-9f1514a200d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:52.744703 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f1123-1a72-45b9-8eb5-9f1514a200d7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:52.744986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f1123-1a72-45b9-8eb5-9f1514a200d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:52.745261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f1123-1a72-45b9-8eb5-9f1514a200d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:52.746077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f1123-1a72-45b9-8eb5-9f1514a200d7 None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=e175b8de037d45d48cbddd64e552f6f2) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:52.746652 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f1123-1a72-45b9-8eb5-9f1514a200d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:52.749287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f1123-1a72-45b9-8eb5-9f1514a200d7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:52.766572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 279/541] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:08:52 2022] GET /identity/v3/role_assignments?user.id=e175b8de037d45d48cbddd64e552f6f2&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:52.768023 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f1123-1a72-45b9-8eb5-9f1514a200d7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:52.919289 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e166d31-d0a4-4f39-a341-57f47af19c66 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:52.965627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13fe77-7b00-4cd4-9006-4ae2ac31ae81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:52.966762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13fe77-7b00-4cd4-9006-4ae2ac31ae81 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:52.967242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13fe77-7b00-4cd4-9006-4ae2ac31ae81 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:52.967709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13fe77-7b00-4cd4-9006-4ae2ac31ae81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:52.968670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13fe77-7b00-4cd4-9006-4ae2ac31ae81 None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=e9fb56e56c7f49ce959d0bc6c515044c) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:52.969220 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13fe77-7b00-4cd4-9006-4ae2ac31ae81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:52.971596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13fe77-7b00-4cd4-9006-4ae2ac31ae81 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:52.983880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 263/542] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:08:52 2022] GET /identity/v3/role_assignments?user.id=e9fb56e56c7f49ce959d0bc6c515044c&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:53.068618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13fe77-7b00-4cd4-9006-4ae2ac31ae81 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:53.117455 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f1123-1a72-45b9-8eb5-9f1514a200d7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:53.140276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:53.180133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b411e-e80d-482f-ace1-434e727129cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:53.180701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b411e-e80d-482f-ace1-434e727129cd None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:53.180985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b411e-e80d-482f-ace1-434e727129cd None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:53.181290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b411e-e80d-482f-ace1-434e727129cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:53.182141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b411e-e80d-482f-ace1-434e727129cd None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=d2d76fb9cbdd40f9b2a748209cee5b75) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:53.182464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b411e-e80d-482f-ace1-434e727129cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:53.184044 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b411e-e80d-482f-ace1-434e727129cd None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:53.202001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 280/543] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:08:52 2022] GET /identity/v3/role_assignments?user.id=d2d76fb9cbdd40f9b2a748209cee5b75&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 259 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:53.234683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:53.257601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13fe77-7b00-4cd4-9006-4ae2ac31ae81 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:53.297063 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9e2b7-18b6-4eb6-9a7b-e2833870dbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:53.297781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9e2b7-18b6-4eb6-9a7b-e2833870dbab None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:53.298081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9e2b7-18b6-4eb6-9a7b-e2833870dbab None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:53.298359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c9e2b7-18b6-4eb6-9a7b-e2833870dbab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:53.312175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c9e2b7-18b6-4eb6-9a7b-e2833870dbab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-94c9e2b7-18b6-4eb6-9a7b-e2833870dbab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:53.317913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:53.319635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:53.322031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 264/544] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:08:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:08:53.329027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b411e-e80d-482f-ace1-434e727129cd None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:53.463778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b411e-e80d-482f-ace1-434e727129cd None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:53.528637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d15a63-95b8-497f-af8f-6856a0d15e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:53.529634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d15a63-95b8-497f-af8f-6856a0d15e1f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:53.530152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d15a63-95b8-497f-af8f-6856a0d15e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:53.530738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d15a63-95b8-497f-af8f-6856a0d15e1f None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:53.532003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d15a63-95b8-497f-af8f-6856a0d15e1f None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:53.532564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d15a63-95b8-497f-af8f-6856a0d15e1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:53.535239 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d15a63-95b8-497f-af8f-6856a0d15e1f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:53.558815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 281/545] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:08:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:53.564788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9e2b7-18b6-4eb6-9a7b-e2833870dbab None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:53.782692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c9e2b7-18b6-4eb6-9a7b-e2833870dbab None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:53.784910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31d7a9-2f95-46e1-afe4-5434b7b0d721 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:53.785695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31d7a9-2f95-46e1-afe4-5434b7b0d721 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:53.786204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31d7a9-2f95-46e1-afe4-5434b7b0d721 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:53.788115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 282/546] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:08:53.794641 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f3d01-12ab-4ba1-809f-7f4cc03bdcd9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:53.795258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f3d01-12ab-4ba1-809f-7f4cc03bdcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:53.795890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f3d01-12ab-4ba1-809f-7f4cc03bdcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:53.830322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6b9f3d01-12ab-4ba1-809f-7f4cc03bdcd9 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:53.832636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6b9f3d01-12ab-4ba1-809f-7f4cc03bdcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:53.886947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3038e-264b-4daf-b41c-9f4742f5244f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:53.888110 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3038e-264b-4daf-b41c-9f4742f5244f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:53.888621 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3038e-264b-4daf-b41c-9f4742f5244f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:53.889132 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3038e-264b-4daf-b41c-9f4742f5244f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:53.906562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:53.916840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3038e-264b-4daf-b41c-9f4742f5244f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:53.920942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3038e-264b-4daf-b41c-9f4742f5244f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-47d3038e-264b-4daf-b41c-9f4742f5244f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:53.954318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:53.955342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:53.956179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:53.956179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:53.956179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:53.956964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 265/547] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:53.964094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3038e-264b-4daf-b41c-9f4742f5244f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:53.966509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:54.083914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 283/548] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 21 16:08:54.174418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3038e-264b-4daf-b41c-9f4742f5244f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:54.225217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00267c68-20ec-4e64-b234-2c2216464fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:54.225845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00267c68-20ec-4e64-b234-2c2216464fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:54.226154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00267c68-20ec-4e64-b234-2c2216464fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:54.226434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00267c68-20ec-4e64-b234-2c2216464fa0 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:54.227104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00267c68-20ec-4e64-b234-2c2216464fa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:54.227415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00267c68-20ec-4e64-b234-2c2216464fa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:54.229405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00267c68-20ec-4e64-b234-2c2216464fa0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:54.248659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 266/549] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:54.259226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f3d01-12ab-4ba1-809f-7f4cc03bdcd9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:54.410824 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00267c68-20ec-4e64-b234-2c2216464fa0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:54.444550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f3d01-12ab-4ba1-809f-7f4cc03bdcd9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:54.499071 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:54.517503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a5fb3-e50f-4b6f-80d9-002201e78175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:54.518547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a5fb3-e50f-4b6f-80d9-002201e78175 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:54.519028 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a5fb3-e50f-4b6f-80d9-002201e78175 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:54.519636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a5fb3-e50f-4b6f-80d9-002201e78175 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:54.531331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a5fb3-e50f-4b6f-80d9-002201e78175 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:54.533463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a5fb3-e50f-4b6f-80d9-002201e78175 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-836a5fb3-e50f-4b6f-80d9-002201e78175 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:08:54.549502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:54.550502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:54.551390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:54.551390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:54.551390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:08:54.551390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:08:54.551390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:08:54.551390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:08:54.551390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:08:54.551390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:08:54.551390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 16:08:54.551390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:54.553967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 284/550] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:08:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:54.562318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a5fb3-e50f-4b6f-80d9-002201e78175 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:54.624630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:54.645524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00267c68-20ec-4e64-b234-2c2216464fa0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:54.730498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbab6b-aa90-442a-97c4-4c1872be14c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:54.731480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbab6b-aa90-442a-97c4-4c1872be14c4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:54.731991 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbab6b-aa90-442a-97c4-4c1872be14c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:54.732456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdbab6b-aa90-442a-97c4-4c1872be14c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:54.753237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdbab6b-aa90-442a-97c4-4c1872be14c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-efdbab6b-aa90-442a-97c4-4c1872be14c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:54.762832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:54.763922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:54.764734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:54.764734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:54.764734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:54.764734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:54.764734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:54.764734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:54.764734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:54.764734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:54.764734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:54.764734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:54.764734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:54.764734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:54.770000 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 267/551] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:08:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:54.774626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbab6b-aa90-442a-97c4-4c1872be14c4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:54.788374 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a5fb3-e50f-4b6f-80d9-002201e78175 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:54.891871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ac154-3d73-4799-afcc-7ca9d8f45590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:54.892510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ac154-3d73-4799-afcc-7ca9d8f45590 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:54.893135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ac154-3d73-4799-afcc-7ca9d8f45590 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:54.893135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ac154-3d73-4799-afcc-7ca9d8f45590 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:54.893821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ac154-3d73-4799-afcc-7ca9d8f45590 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:54.894165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ac154-3d73-4799-afcc-7ca9d8f45590 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:54.895629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ac154-3d73-4799-afcc-7ca9d8f45590 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:54.896885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdbab6b-aa90-442a-97c4-4c1872be14c4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:54.910300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:54.940498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d7e80-c015-4476-85b7-e7490ce78b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:54.941526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d7e80-c015-4476-85b7-e7490ce78b9f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:54.942077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d7e80-c015-4476-85b7-e7490ce78b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:54.942658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d7e80-c015-4476-85b7-e7490ce78b9f None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:54.943540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d7e80-c015-4476-85b7-e7490ce78b9f None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:54.944060 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d7e80-c015-4476-85b7-e7490ce78b9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:54.945622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d7e80-c015-4476-85b7-e7490ce78b9f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:54.956512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 268/552] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:08:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:54.969361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d7e80-c015-4476-85b7-e7490ce78b9f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:54.975231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 285/553] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Sep 21 16:08:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:08:54.988338 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ac154-3d73-4799-afcc-7ca9d8f45590 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:55.157531 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d7e80-c015-4476-85b7-e7490ce78b9f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:55.210408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:55.221522 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7dfa8-3bf8-4fd5-aa40-b2bb7cb96448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:55.222185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7dfa8-3bf8-4fd5-aa40-b2bb7cb96448 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:55.222465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7dfa8-3bf8-4fd5-aa40-b2bb7cb96448 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:55.222733 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad7dfa8-3bf8-4fd5-aa40-b2bb7cb96448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:55.226935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ac154-3d73-4799-afcc-7ca9d8f45590 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:55.257612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7dfa8-3bf8-4fd5-aa40-b2bb7cb96448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:55.260341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad7dfa8-3bf8-4fd5-aa40-b2bb7cb96448 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-bad7dfa8-3bf8-4fd5-aa40-b2bb7cb96448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:55.297145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:55.298246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:55.299481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:55.299481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:55.299481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:55.299669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 269/554] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:08:55.308253 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7dfa8-3bf8-4fd5-aa40-b2bb7cb96448 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:55.331445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f790a97-639d-412d-8e2b-557a0c842de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:55.332485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f790a97-639d-412d-8e2b-557a0c842de2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:55.332992 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f790a97-639d-412d-8e2b-557a0c842de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:55.333482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f790a97-639d-412d-8e2b-557a0c842de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:55.356263 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:55.540601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f790a97-639d-412d-8e2b-557a0c842de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:55.543384 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f790a97-639d-412d-8e2b-557a0c842de2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:55.584418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad7dfa8-3bf8-4fd5-aa40-b2bb7cb96448 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:55.709338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d01f6-b77d-4304-9a5d-225b1ec1ad9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:55.710894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d01f6-b77d-4304-9a5d-225b1ec1ad9f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:55.711635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d01f6-b77d-4304-9a5d-225b1ec1ad9f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:55.712965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d01f6-b77d-4304-9a5d-225b1ec1ad9f None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:55.714406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d01f6-b77d-4304-9a5d-225b1ec1ad9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:55.715221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d01f6-b77d-4304-9a5d-225b1ec1ad9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:55.718093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d01f6-b77d-4304-9a5d-225b1ec1ad9f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:55.764245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 270/555] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 456 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:55.770793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d01f6-b77d-4304-9a5d-225b1ec1ad9f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:55.786516 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:55.814407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 286/556] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:08:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 823 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:08:55.820463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f790a97-639d-412d-8e2b-557a0c842de2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:56.042584 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:56.079103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d01f6-b77d-4304-9a5d-225b1ec1ad9f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:56.112914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f790a97-639d-412d-8e2b-557a0c842de2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:56.147168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb209f2d-72d8-4152-8e2d-f2fa7e157589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:56.148708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb209f2d-72d8-4152-8e2d-f2fa7e157589 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:56.149244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb209f2d-72d8-4152-8e2d-f2fa7e157589 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:56.149755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb209f2d-72d8-4152-8e2d-f2fa7e157589 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:56.163568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb209f2d-72d8-4152-8e2d-f2fa7e157589 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:56.167608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb209f2d-72d8-4152-8e2d-f2fa7e157589 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:56.176737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a8f34-cfed-4a38-a8e6-d3f346d8c441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:56.177734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a8f34-cfed-4a38-a8e6-d3f346d8c441 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:56.178266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a8f34-cfed-4a38-a8e6-d3f346d8c441 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:56.178655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a8f34-cfed-4a38-a8e6-d3f346d8c441 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-cb209f2d-72d8-4152-8e2d-f2fa7e157589 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:08:56.191167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:56.192180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:56.193103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:56.193103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:56.193103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:08:56.193103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:08:56.193103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:08:56.193103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:08:56.193103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:08:56.193103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:08:56.193103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:08:56.193103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:56.194300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 271/557] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Sep 21 16:08:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 424 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:56.197554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab810e-0291-400b-bf83-1a288606b555 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:56.198307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab810e-0291-400b-bf83-1a288606b555 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:56.198651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a8f34-cfed-4a38-a8e6-d3f346d8c441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:56.198823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab810e-0291-400b-bf83-1a288606b555 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:56.201350 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396a8f34-cfed-4a38-a8e6-d3f346d8c441 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:56.201755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 272/558] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:56.203485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab810e-0291-400b-bf83-1a288606b555 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-396a8f34-cfed-4a38-a8e6-d3f346d8c441 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:56.232827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:56.233926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:56.234873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:56.234873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:56.234873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:56.235481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 287/559] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:08:56.238092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd748e95-beb9-4707-9bac-ed3fcae70b1a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:56.238092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd748e95-beb9-4707-9bac-ed3fcae70b1a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:56.238092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd748e95-beb9-4707-9bac-ed3fcae70b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:56.265338 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-cd748e95-beb9-4707-9bac-ed3fcae70b1a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:56.266613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cd748e95-beb9-4707-9bac-ed3fcae70b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:56.323483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:56.345024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab810e-0291-400b-bf83-1a288606b555 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:56.382463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 288/560] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:56.384843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1197bc-a841-4b23-8e1a-08723d2ff44a None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:56.385306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1197bc-a841-4b23-8e1a-08723d2ff44a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:56.385700 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1197bc-a841-4b23-8e1a-08723d2ff44a None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:56.394238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 289/561] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:56.394238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1197bc-a841-4b23-8e1a-08723d2ff44a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:56.403767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6131948-f751-41ec-be12-4ba17825d110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:56.404575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6131948-f751-41ec-be12-4ba17825d110 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:56.404955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6131948-f751-41ec-be12-4ba17825d110 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:56.405332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6131948-f751-41ec-be12-4ba17825d110 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:56.406291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6131948-f751-41ec-be12-4ba17825d110 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:56.406670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6131948-f751-41ec-be12-4ba17825d110 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:56.408248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6131948-f751-41ec-be12-4ba17825d110 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:56.431977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 273/562] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Sep 21 16:08:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:56.433593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba9a59-5d55-42a5-b91f-6642ed5301a5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:56.433961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba9a59-5d55-42a5-b91f-6642ed5301a5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:56.434329 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba9a59-5d55-42a5-b91f-6642ed5301a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:56.441740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:56.457919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-64ba9a59-5d55-42a5-b91f-6642ed5301a5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:56.459651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-64ba9a59-5d55-42a5-b91f-6642ed5301a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:56.552295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 274/563] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:56.553488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba9a59-5d55-42a5-b91f-6642ed5301a5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:56.587347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1197bc-a841-4b23-8e1a-08723d2ff44a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:56.646108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860f519-84ab-4755-9ecf-196c51ce1b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:56.647076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860f519-84ab-4755-9ecf-196c51ce1b67 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:56.647561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860f519-84ab-4755-9ecf-196c51ce1b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:56.648035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860f519-84ab-4755-9ecf-196c51ce1b67 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:56.649185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860f519-84ab-4755-9ecf-196c51ce1b67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:56.649722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860f519-84ab-4755-9ecf-196c51ce1b67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:56.652588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1860f519-84ab-4755-9ecf-196c51ce1b67 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:56.683302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 290/564] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:56.684233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee8b8e-e4f6-4da8-a2ce-190400690e14 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:56.684718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee8b8e-e4f6-4da8-a2ce-190400690e14 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:56.685193 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee8b8e-e4f6-4da8-a2ce-190400690e14 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:56.688413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 291/565] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:08:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:08:56.689915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee8b8e-e4f6-4da8-a2ce-190400690e14 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:56.702534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba9a59-5d55-42a5-b91f-6642ed5301a5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:56.736541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:56.743148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bc15e-e740-41b4-837c-85da42d595d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:56.743858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bc15e-e740-41b4-837c-85da42d595d9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:56.744140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bc15e-e740-41b4-837c-85da42d595d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:56.744421 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bc15e-e740-41b4-837c-85da42d595d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:56.748248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:56.756466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bc15e-e740-41b4-837c-85da42d595d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:56.757960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bc15e-e740-41b4-837c-85da42d595d9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-8e9bc15e-e740-41b4-837c-85da42d595d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:56.774423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:56.776036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:56.776923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:56.776923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:56.776923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:56.781019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 275/566] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:56.782451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836be21-6039-41a6-9cc6-e7f5734dbba3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:56.782742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836be21-6039-41a6-9cc6-e7f5734dbba3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:56.783016 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836be21-6039-41a6-9cc6-e7f5734dbba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:56.801814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a836be21-6039-41a6-9cc6-e7f5734dbba3 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:08:56.803070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a836be21-6039-41a6-9cc6-e7f5734dbba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:08:56.876589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee8b8e-e4f6-4da8-a2ce-190400690e14 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:56.890293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 276/567] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:08:56.890293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836be21-6039-41a6-9cc6-e7f5734dbba3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:56.945650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497c893-4d6d-4e24-97ff-0d2d2358e4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:56.947213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497c893-4d6d-4e24-97ff-0d2d2358e4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:56.948888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497c893-4d6d-4e24-97ff-0d2d2358e4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:56.948888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497c893-4d6d-4e24-97ff-0d2d2358e4b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:56.959989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497c893-4d6d-4e24-97ff-0d2d2358e4b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:56.963214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1497c893-4d6d-4e24-97ff-0d2d2358e4b3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-1497c893-4d6d-4e24-97ff-0d2d2358e4b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:08:56.984513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:56.985471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:56.986381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:08:56.986381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:08:56.986381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:08:56.986381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:08:56.986381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:08:56.986381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:08:56.986381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:08:56.986381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:56.986790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 292/568] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:08:56.988238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497c893-4d6d-4e24-97ff-0d2d2358e4b3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:57.067587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a836be21-6039-41a6-9cc6-e7f5734dbba3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:57.087677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:57.120340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9d6e8-9c11-48f2-a246-1a48ad6d4794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:57.121329 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9d6e8-9c11-48f2-a246-1a48ad6d4794 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:57.121769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9d6e8-9c11-48f2-a246-1a48ad6d4794 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:57.122201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9d6e8-9c11-48f2-a246-1a48ad6d4794 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:57.129079 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:57.139661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9d6e8-9c11-48f2-a246-1a48ad6d4794 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-dbb9d6e8-9c11-48f2-a246-1a48ad6d4794 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:57.148246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:57.150232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:57.151795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 277/569] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:08:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:57.153118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497c893-4d6d-4e24-97ff-0d2d2358e4b3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:57.158277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9d6e8-9c11-48f2-a246-1a48ad6d4794 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:57.191811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2506e64-c249-4892-ad15-8d8bc5ab2398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:57.192428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2506e64-c249-4892-ad15-8d8bc5ab2398 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:57.192698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2506e64-c249-4892-ad15-8d8bc5ab2398 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:57.192962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2506e64-c249-4892-ad15-8d8bc5ab2398 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:57.193605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2506e64-c249-4892-ad15-8d8bc5ab2398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:57.193900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2506e64-c249-4892-ad15-8d8bc5ab2398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:57.195297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2506e64-c249-4892-ad15-8d8bc5ab2398 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:57.209229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 293/570] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:57.210008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2506e64-c249-4892-ad15-8d8bc5ab2398 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:57.271792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9d6e8-9c11-48f2-a246-1a48ad6d4794 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:57.306885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21bc56-cfea-468c-aa6e-071c11f6d425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:57.307519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21bc56-cfea-468c-aa6e-071c11f6d425 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:57.307826 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21bc56-cfea-468c-aa6e-071c11f6d425 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:57.308093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21bc56-cfea-468c-aa6e-071c11f6d425 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:57.318798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21bc56-cfea-468c-aa6e-071c11f6d425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ef21bc56-cfea-468c-aa6e-071c11f6d425 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:57.324248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:57.325678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:57.326983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 278/571] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:08:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:57.328351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21bc56-cfea-468c-aa6e-071c11f6d425 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:57.377344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2506e64-c249-4892-ad15-8d8bc5ab2398 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:57.382212 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:57.434465 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596090b3-3654-4d72-a228-ea2073d9e962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:57.435414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596090b3-3654-4d72-a228-ea2073d9e962 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:57.435865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596090b3-3654-4d72-a228-ea2073d9e962 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:57.436304 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596090b3-3654-4d72-a228-ea2073d9e962 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:57.437462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596090b3-3654-4d72-a228-ea2073d9e962 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:57.437962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596090b3-3654-4d72-a228-ea2073d9e962 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:57.440227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596090b3-3654-4d72-a228-ea2073d9e962 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:57.467423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 294/572] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:08:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:57.467423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596090b3-3654-4d72-a228-ea2073d9e962 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:57.481595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:57.500032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21bc56-cfea-468c-aa6e-071c11f6d425 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:57.562957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f5635-ed09-4ba4-a395-ce6e50efde65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:57.564063 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f5635-ed09-4ba4-a395-ce6e50efde65 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:57.564548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f5635-ed09-4ba4-a395-ce6e50efde65 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:57.565045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f5635-ed09-4ba4-a395-ce6e50efde65 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:57.565936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f5635-ed09-4ba4-a395-ce6e50efde65 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:57.566333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f5635-ed09-4ba4-a395-ce6e50efde65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:57.568633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f5635-ed09-4ba4-a395-ce6e50efde65 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:57.590463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 279/573] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:08:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:57.590463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f5635-ed09-4ba4-a395-ce6e50efde65 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:57.596531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596090b3-3654-4d72-a228-ea2073d9e962 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:57.652806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd3fa1-d0be-4a9c-a735-e803ad076a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:57.653716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd3fa1-d0be-4a9c-a735-e803ad076a73 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:57.654123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd3fa1-d0be-4a9c-a735-e803ad076a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:57.654482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd3fa1-d0be-4a9c-a735-e803ad076a73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:57.671107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dd3fa1-d0be-4a9c-a735-e803ad076a73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-82dd3fa1-d0be-4a9c-a735-e803ad076a73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:08:57.680490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:57.682415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:57.683347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:57.683347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:57.683347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:57.683347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:57.683347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:08:57.683347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:08:57.683347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:08:57.683347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:08:57.683347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:08:57.683347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:08:57.683347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:08:57.683347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:57.684004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 295/574] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:08:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:57.684925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd3fa1-d0be-4a9c-a735-e803ad076a73 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:57.711710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:57.719675 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f5635-ed09-4ba4-a395-ce6e50efde65 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:57.761305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aba4e-ec58-49da-9575-0a89ecf1330f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:57.761942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aba4e-ec58-49da-9575-0a89ecf1330f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:57.762276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aba4e-ec58-49da-9575-0a89ecf1330f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:57.762557 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aba4e-ec58-49da-9575-0a89ecf1330f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:57.769881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aba4e-ec58-49da-9575-0a89ecf1330f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:57.771370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02aba4e-ec58-49da-9575-0a89ecf1330f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:57.777967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b02aba4e-ec58-49da-9575-0a89ecf1330f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:08:57.785769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:57.793436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:57.794479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:08:57.794479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:08:57.794479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:08:57.794479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:08:57.794479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:08:57.794479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:08:57.794479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:08:57.794479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:57.794479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 280/575] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:08:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:57.794479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aba4e-ec58-49da-9575-0a89ecf1330f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:57.829081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd3fa1-d0be-4a9c-a735-e803ad076a73 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:57.873959 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8caccaa-14d5-46a1-ac9c-8d012c337684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:57.874998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8caccaa-14d5-46a1-ac9c-8d012c337684 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:57.875501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8caccaa-14d5-46a1-ac9c-8d012c337684 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:57.875981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8caccaa-14d5-46a1-ac9c-8d012c337684 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:57.876871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8caccaa-14d5-46a1-ac9c-8d012c337684 None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:57.877406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8caccaa-14d5-46a1-ac9c-8d012c337684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:57.880009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8caccaa-14d5-46a1-ac9c-8d012c337684 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:57.891078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 296/576] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:08:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:57.892006 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8caccaa-14d5-46a1-ac9c-8d012c337684 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:57.913702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02aba4e-ec58-49da-9575-0a89ecf1330f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:57.956992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc09b64-fcd8-4aa5-8305-285c941fde94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:57.957858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc09b64-fcd8-4aa5-8305-285c941fde94 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:57.958248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc09b64-fcd8-4aa5-8305-285c941fde94 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:57.958649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc09b64-fcd8-4aa5-8305-285c941fde94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:57.959473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc09b64-fcd8-4aa5-8305-285c941fde94 None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=d71a1b8081ce490db46db95f14024ba6) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:57.959814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc09b64-fcd8-4aa5-8305-285c941fde94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:57.961672 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc09b64-fcd8-4aa5-8305-285c941fde94 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:57.968862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 281/577] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:08:57 2022] GET /identity/v3/role_assignments?user.id=d71a1b8081ce490db46db95f14024ba6&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:57.970456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc09b64-fcd8-4aa5-8305-285c941fde94 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:58.010087 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:58.029268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8caccaa-14d5-46a1-ac9c-8d012c337684 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:58.075616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3a6fa-b003-4539-91ce-f10cd0d776b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:58.076837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3a6fa-b003-4539-91ce-f10cd0d776b8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:58.077357 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3a6fa-b003-4539-91ce-f10cd0d776b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:58.077856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3a6fa-b003-4539-91ce-f10cd0d776b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:58.081418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:58.094839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3a6fa-b003-4539-91ce-f10cd0d776b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:58.096641 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3a6fa-b003-4539-91ce-f10cd0d776b8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-eec3a6fa-b003-4539-91ce-f10cd0d776b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:58.115952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:58.117944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:58.119926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:58.119926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:58.119926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:58.123628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 297/578] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:58.124932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3a6fa-b003-4539-91ce-f10cd0d776b8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:58.158770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc09b64-fcd8-4aa5-8305-285c941fde94 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:58.214854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22837afc-c3c5-4936-934c-074cd78dfe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:58.215708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22837afc-c3c5-4936-934c-074cd78dfe86 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:58.216112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22837afc-c3c5-4936-934c-074cd78dfe86 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:58.216510 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22837afc-c3c5-4936-934c-074cd78dfe86 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:58.217321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22837afc-c3c5-4936-934c-074cd78dfe86 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:58.217746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22837afc-c3c5-4936-934c-074cd78dfe86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:58.219367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22837afc-c3c5-4936-934c-074cd78dfe86 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:58.231287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 282/579] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:08:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:58.232876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22837afc-c3c5-4936-934c-074cd78dfe86 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:58.317051 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3a6fa-b003-4539-91ce-f10cd0d776b8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:58.343871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:58.377184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22837afc-c3c5-4936-934c-074cd78dfe86 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:58.379879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f239-1dc7-484b-9612-36855cde18d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:58.380837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f239-1dc7-484b-9612-36855cde18d6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:58.382086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f239-1dc7-484b-9612-36855cde18d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:58.382086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f239-1dc7-484b-9612-36855cde18d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:58.382380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f239-1dc7-484b-9612-36855cde18d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:58.382862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f239-1dc7-484b-9612-36855cde18d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:58.384934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f239-1dc7-484b-9612-36855cde18d6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:58.407594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 298/580] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:08:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:58.409511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f239-1dc7-484b-9612-36855cde18d6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:58.431968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e130fb4-41d4-41b4-abef-af71cdc18627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:58.432840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e130fb4-41d4-41b4-abef-af71cdc18627 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:58.433283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e130fb4-41d4-41b4-abef-af71cdc18627 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:58.433681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e130fb4-41d4-41b4-abef-af71cdc18627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:58.448033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e130fb4-41d4-41b4-abef-af71cdc18627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:58.450007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e130fb4-41d4-41b4-abef-af71cdc18627 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0e130fb4-41d4-41b4-abef-af71cdc18627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:58.468978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:58.470832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:58.473955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:58.473955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:58.473955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:58.473955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 283/581] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:58.473955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e130fb4-41d4-41b4-abef-af71cdc18627 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:58.505210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:58.608591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f239-1dc7-484b-9612-36855cde18d6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:58.610007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e130fb4-41d4-41b4-abef-af71cdc18627 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:58.649063 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fb061-00a9-42d6-a579-e2571bdd946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:58.650372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fb061-00a9-42d6-a579-e2571bdd946e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:58.650653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fb061-00a9-42d6-a579-e2571bdd946e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:58.650925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fb061-00a9-42d6-a579-e2571bdd946e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:58.653239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:58.663274 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fb061-00a9-42d6-a579-e2571bdd946e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:58.664739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fb061-00a9-42d6-a579-e2571bdd946e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:58.674675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344449a-2eb5-4357-aed4-ee33f11294ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:58.675985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344449a-2eb5-4357-aed4-ee33f11294ed None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:58.676672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344449a-2eb5-4357-aed4-ee33f11294ed None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:58.677381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344449a-2eb5-4357-aed4-ee33f11294ed None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:58.678791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344449a-2eb5-4357-aed4-ee33f11294ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:58.679533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344449a-2eb5-4357-aed4-ee33f11294ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7f1fb061-00a9-42d6-a579-e2571bdd946e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:08:58.680258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:58.682720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:08:58.684444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:08:58.684444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:08:58.684444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:58.684787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c344449a-2eb5-4357-aed4-ee33f11294ed None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:58.686988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 284/582] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:58.687921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fb061-00a9-42d6-a579-e2571bdd946e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:58.699160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 299/583] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:58.700015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344449a-2eb5-4357-aed4-ee33f11294ed None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:58.822985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344449a-2eb5-4357-aed4-ee33f11294ed None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:58.828173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fb061-00a9-42d6-a579-e2571bdd946e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:58.833080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:58.866013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8d320-34d9-452d-a338-d7e4c3603a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:58.866668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8d320-34d9-452d-a338-d7e4c3603a9d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:58.866948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8d320-34d9-452d-a338-d7e4c3603a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:58.867225 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8d320-34d9-452d-a338-d7e4c3603a9d None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:58.867902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8d320-34d9-452d-a338-d7e4c3603a9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:58.868189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8d320-34d9-452d-a338-d7e4c3603a9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:58.869681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8d320-34d9-452d-a338-d7e4c3603a9d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:58.874226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ea24a-7901-4ea6-9f08-ff16e4f44f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:58.875119 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ea24a-7901-4ea6-9f08-ff16e4f44f32 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:58.875570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ea24a-7901-4ea6-9f08-ff16e4f44f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:58.876003 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ea24a-7901-4ea6-9f08-ff16e4f44f32 None admin] PATH_INFO: `/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/f0acf23d224e4b16ad03cdd9bf7cae8e/roles/5f8e9da552194885bbd7684f1a9c4f2d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:58.887336 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 300/584] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:58.887336 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8d320-34d9-452d-a338-d7e4c3603a9d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:58.906785 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ea24a-7901-4ea6-9f08-ff16e4f44f32 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9e1d9331b4d4dbd3dd447937c42ad, user_id=f0acf23d224e4b16ad03cdd9bf7cae8e, role_id=5f8e9da552194885bbd7684f1a9c4f2d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:58.908141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21ea24a-7901-4ea6-9f08-ff16e4f44f32 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:58.924291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 285/585] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:08:58 2022] PUT /identity/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/f0acf23d224e4b16ad03cdd9bf7cae8e/roles/5f8e9da552194885bbd7684f1a9c4f2d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:08:58.931877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ea24a-7901-4ea6-9f08-ff16e4f44f32 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:59.029151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:59.052633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8d320-34d9-452d-a338-d7e4c3603a9d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:59.102224 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091cede-cc1a-4678-8231-f7523305e552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:59.102888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091cede-cc1a-4678-8231-f7523305e552 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:59.103170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091cede-cc1a-4678-8231-f7523305e552 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:59.103432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091cede-cc1a-4678-8231-f7523305e552 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:59.104122 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091cede-cc1a-4678-8231-f7523305e552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:59.104484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091cede-cc1a-4678-8231-f7523305e552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:59.105986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091cede-cc1a-4678-8231-f7523305e552 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:59.122038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 301/586] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:59.123027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091cede-cc1a-4678-8231-f7523305e552 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:59.166803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ea24a-7901-4ea6-9f08-ff16e4f44f32 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:59.172727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:59.207420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831a0a9-39c4-4185-948f-115ed6737023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:59.208055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831a0a9-39c4-4185-948f-115ed6737023 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:59.208328 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831a0a9-39c4-4185-948f-115ed6737023 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:59.208594 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831a0a9-39c4-4185-948f-115ed6737023 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:59.217841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831a0a9-39c4-4185-948f-115ed6737023 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:59.219398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7831a0a9-39c4-4185-948f-115ed6737023 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7831a0a9-39c4-4185-948f-115ed6737023 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:08:59.233618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:59.235465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:59.237307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:59.237307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:59.237307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:08:59.237307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:08:59.237307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:08:59.237307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:08:59.237307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:08:59.237307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:08:59.237307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 16:08:59.237307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:59.241933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 286/587] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:59.241933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831a0a9-39c4-4185-948f-115ed6737023 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:59.340259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091cede-cc1a-4678-8231-f7523305e552 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:59.357425 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:59.363884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831a0a9-39c4-4185-948f-115ed6737023 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:59.403592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0f2af-7560-4f64-a3c4-3ba12b9c5d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:59.404257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0f2af-7560-4f64-a3c4-3ba12b9c5d5f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:59.404585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0f2af-7560-4f64-a3c4-3ba12b9c5d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:59.404862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0f2af-7560-4f64-a3c4-3ba12b9c5d5f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:59.412094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e4cde-43c3-4f3c-824e-d51f50003491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:59.413540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e4cde-43c3-4f3c-824e-d51f50003491 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:59.413842 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0f2af-7560-4f64-a3c4-3ba12b9c5d5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:59.414504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e4cde-43c3-4f3c-824e-d51f50003491 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:59.415192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e4cde-43c3-4f3c-824e-d51f50003491 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:59.416145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0f2af-7560-4f64-a3c4-3ba12b9c5d5f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:59.430444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e4cde-43c3-4f3c-824e-d51f50003491 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-2bd0f2af-7560-4f64-a3c4-3ba12b9c5d5f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:08:59.430862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:59.432419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:59.434124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:59.434124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:59.434124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:08:59.434124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:08:59.434124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:08:59.434124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:08:59.434124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:08:59.434124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:08:59.434124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 21 16:08:59.434124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:08:59.437028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 287/588] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:08:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:59.438177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0f2af-7560-4f64-a3c4-3ba12b9c5d5f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:59.439183 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e4cde-43c3-4f3c-824e-d51f50003491 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-598e4cde-43c3-4f3c-824e-d51f50003491 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:08:59.466177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:08:59.468232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:08:59.470269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:08:59.470269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:08:59.470269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:08:59.470269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:08:59.470269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:08:59.470269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:08:59.470269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:08:59.470269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:08:59.470269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 16:08:59.470269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:08:59.474972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 302/589] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:08:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:08:59.477062 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e4cde-43c3-4f3c-824e-d51f50003491 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:59.565409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0f2af-7560-4f64-a3c4-3ba12b9c5d5f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:59.608806 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812606a-0200-4180-b48d-d957580e1cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:59.609348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812606a-0200-4180-b48d-d957580e1cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:59.609624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812606a-0200-4180-b48d-d957580e1cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:59.609894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812606a-0200-4180-b48d-d957580e1cc1 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:59.610679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812606a-0200-4180-b48d-d957580e1cc1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:59.611093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812606a-0200-4180-b48d-d957580e1cc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:59.612429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812606a-0200-4180-b48d-d957580e1cc1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:59.640499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 288/590] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Sep 21 16:08:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:59.641567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812606a-0200-4180-b48d-d957580e1cc1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:59.646817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:59.664679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:59.673070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e4cde-43c3-4f3c-824e-d51f50003491 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:59.745402 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf66a8-1500-496c-a881-12d878fa20d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:59.746554 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf66a8-1500-496c-a881-12d878fa20d8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:59.747107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf66a8-1500-496c-a881-12d878fa20d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:59.747915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf66a8-1500-496c-a881-12d878fa20d8 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:59.748973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf66a8-1500-496c-a881-12d878fa20d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:59.749474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf66a8-1500-496c-a881-12d878fa20d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:59.752736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf66a8-1500-496c-a881-12d878fa20d8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:59.780149 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812606a-0200-4180-b48d-d957580e1cc1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:59.792616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 303/591] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Sep 21 16:08:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:59.792616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf66a8-1500-496c-a881-12d878fa20d8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:59.830878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b3924-ae0f-4b4b-8681-5a040de9c8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:08:59.831761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b3924-ae0f-4b4b-8681-5a040de9c8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:08:59.832216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b3924-ae0f-4b4b-8681-5a040de9c8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:08:59.832668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8b3924-ae0f-4b4b-8681-5a040de9c8c6 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:08:59.833648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b3924-ae0f-4b4b-8681-5a040de9c8c6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:08:59.834150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b3924-ae0f-4b4b-8681-5a040de9c8c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:08:59.835788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8b3924-ae0f-4b4b-8681-5a040de9c8c6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:08:59.861127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 289/592] 10.222.0.64 () {56 vars in 1139 bytes} [Wed Sep 21 16:08:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:08:59.862380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b3924-ae0f-4b4b-8681-5a040de9c8c6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:08:59.952008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:08:59.980155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf66a8-1500-496c-a881-12d878fa20d8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:08:59.991954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8b3924-ae0f-4b4b-8681-5a040de9c8c6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:00.036846 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e57372-583d-44d0-a730-18f86c5fffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:00.037794 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e57372-583d-44d0-a730-18f86c5fffce None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:00.038249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e57372-583d-44d0-a730-18f86c5fffce None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:00.038691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e57372-583d-44d0-a730-18f86c5fffce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:00.045382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effe996-d033-443c-ac03-974a38527979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:00.046670 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effe996-d033-443c-ac03-974a38527979 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:00.047188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effe996-d033-443c-ac03-974a38527979 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:00.047930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effe996-d033-443c-ac03-974a38527979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:00.052797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e57372-583d-44d0-a730-18f86c5fffce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:00.055181 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e57372-583d-44d0-a730-18f86c5fffce None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:00.072155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effe996-d033-443c-ac03-974a38527979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:00.074617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7effe996-d033-443c-ac03-974a38527979 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:00.077827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a2e57372-583d-44d0-a730-18f86c5fffce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:00.086365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:00.087608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:00.088587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:00.088587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:00.088587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:00.091040 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 290/593] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:00.092382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e57372-583d-44d0-a730-18f86c5fffce None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-7effe996-d033-443c-ac03-974a38527979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:00.104345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:00.105344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:00.106258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:00.106258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:00.106258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:00.107718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 304/594] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:08:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:00.108828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effe996-d033-443c-ac03-974a38527979 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:00.241566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effe996-d033-443c-ac03-974a38527979 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:00.303341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3ce09-012f-4f0d-a78e-896327c45c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:00.304365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3ce09-012f-4f0d-a78e-896327c45c6c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:00.304845 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3ce09-012f-4f0d-a78e-896327c45c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:00.305314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3ce09-012f-4f0d-a78e-896327c45c6c None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:00.306822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3ce09-012f-4f0d-a78e-896327c45c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:00.307367 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3ce09-012f-4f0d-a78e-896327c45c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:00.308943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3ce09-012f-4f0d-a78e-896327c45c6c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:00.320565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e57372-583d-44d0-a730-18f86c5fffce None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:00.327244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 305/595] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:00.328541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3ce09-012f-4f0d-a78e-896327c45c6c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:00.386695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:00.397457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:00.413624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c19e0e-94ca-4250-93cd-e642b399d93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:00.414739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c19e0e-94ca-4250-93cd-e642b399d93b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:00.416139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c19e0e-94ca-4250-93cd-e642b399d93b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:00.416651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c19e0e-94ca-4250-93cd-e642b399d93b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:00.448876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c19e0e-94ca-4250-93cd-e642b399d93b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:00.452315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c19e0e-94ca-4250-93cd-e642b399d93b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:00.453721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3ce09-012f-4f0d-a78e-896327c45c6c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-18c19e0e-94ca-4250-93cd-e642b399d93b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:00.490677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:00.491716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:00.492584 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:00.492584 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:00.492584 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:00.493937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 291/596] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:00.495537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c19e0e-94ca-4250-93cd-e642b399d93b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:00.500980 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f8a8a-b180-476c-b120-2eb61d9ab0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:00.501998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f8a8a-b180-476c-b120-2eb61d9ab0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:00.502535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f8a8a-b180-476c-b120-2eb61d9ab0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:00.503035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f8a8a-b180-476c-b120-2eb61d9ab0f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:00.503996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f8a8a-b180-476c-b120-2eb61d9ab0f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:00.504534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f8a8a-b180-476c-b120-2eb61d9ab0f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:00.506217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f8a8a-b180-476c-b120-2eb61d9ab0f1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:00.539737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 306/597] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:00.539737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f8a8a-b180-476c-b120-2eb61d9ab0f1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:00.667194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c19e0e-94ca-4250-93cd-e642b399d93b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:00.703917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f8a8a-b180-476c-b120-2eb61d9ab0f1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:00.708470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d34b5-5d57-4cc0-8822-9c0b175b4159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:00.709118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d34b5-5d57-4cc0-8822-9c0b175b4159 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:00.709403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d34b5-5d57-4cc0-8822-9c0b175b4159 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:00.709686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d34b5-5d57-4cc0-8822-9c0b175b4159 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:00.717797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d34b5-5d57-4cc0-8822-9c0b175b4159 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:00.719305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d34b5-5d57-4cc0-8822-9c0b175b4159 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:00.724071 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1e5d34b5-5d57-4cc0-8822-9c0b175b4159 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:00.732135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:00.733206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:00.734267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:00.734267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:00.734267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:00.734267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:00.734267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:00.734267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:00.734267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:00.734267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:00.738288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 292/598] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:00.739232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d34b5-5d57-4cc0-8822-9c0b175b4159 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:00.749228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dcbe73-996a-4b4e-8ddc-0c4481381c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:00.750379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcbe73-996a-4b4e-8ddc-0c4481381c91 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:00.750660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcbe73-996a-4b4e-8ddc-0c4481381c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:00.750936 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dcbe73-996a-4b4e-8ddc-0c4481381c91 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:00.751982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcbe73-996a-4b4e-8ddc-0c4481381c91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:00.752296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcbe73-996a-4b4e-8ddc-0c4481381c91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:00.754037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dcbe73-996a-4b4e-8ddc-0c4481381c91 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:00.770446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 307/599] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:00.777033 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba66826-ad44-45a1-98da-e2ad96901f6d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:00.777385 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba66826-ad44-45a1-98da-e2ad96901f6d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:00.777618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba66826-ad44-45a1-98da-e2ad96901f6d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:00.779742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 308/600] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:00.779742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba66826-ad44-45a1-98da-e2ad96901f6d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:00.780601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:00.865396 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d34b5-5d57-4cc0-8822-9c0b175b4159 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:00.900155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba66826-ad44-45a1-98da-e2ad96901f6d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:00.913231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83a525-5640-4674-b2e8-f3ff53e9925f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:00.914890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83a525-5640-4674-b2e8-f3ff53e9925f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:00.916390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83a525-5640-4674-b2e8-f3ff53e9925f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:00.916900 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83a525-5640-4674-b2e8-f3ff53e9925f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:00.933542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83a525-5640-4674-b2e8-f3ff53e9925f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:00.936673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c83a525-5640-4674-b2e8-f3ff53e9925f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:00.939599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a346db-e6cd-47c1-a3af-4960d67697e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:00.940266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a346db-e6cd-47c1-a3af-4960d67697e8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:00.940537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a346db-e6cd-47c1-a3af-4960d67697e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:00.940808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a346db-e6cd-47c1-a3af-4960d67697e8 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:00.941507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a346db-e6cd-47c1-a3af-4960d67697e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:00.941816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a346db-e6cd-47c1-a3af-4960d67697e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:00.943302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a346db-e6cd-47c1-a3af-4960d67697e8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:00.958928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 309/601] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:00.958928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a346db-e6cd-47c1-a3af-4960d67697e8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0c83a525-5640-4674-b2e8-f3ff53e9925f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:00.966516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:00.968187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:00.969718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:00.969718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:00.969718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:00.969718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:00.969718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:00.969718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:00.969718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:00.969718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:00.969718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 293/602] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:00.969718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74e6e9-00a5-4c0f-b85b-c89ff583bd26 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:00.969718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74e6e9-00a5-4c0f-b85b-c89ff583bd26 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:00.973907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74e6e9-00a5-4c0f-b85b-c89ff583bd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:01.003327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5c74e6e9-00a5-4c0f-b85b-c89ff583bd26 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:01.005133 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5c74e6e9-00a5-4c0f-b85b-c89ff583bd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:01.032482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:01.126079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 294/603] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:01.126079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74e6e9-00a5-4c0f-b85b-c89ff583bd26 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:01.149362 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a346db-e6cd-47c1-a3af-4960d67697e8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:01.161047 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:01.215636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069495b-04a8-47a2-9f55-9d8ed1f58998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:01.217015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069495b-04a8-47a2-9f55-9d8ed1f58998 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:01.217507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069495b-04a8-47a2-9f55-9d8ed1f58998 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:01.218000 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069495b-04a8-47a2-9f55-9d8ed1f58998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:01.231624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069495b-04a8-47a2-9f55-9d8ed1f58998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:01.234763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069495b-04a8-47a2-9f55-9d8ed1f58998 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:01.251700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74e6e9-00a5-4c0f-b85b-c89ff583bd26 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a069495b-04a8-47a2-9f55-9d8ed1f58998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:01.259417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:01.261170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:01.264553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:01.264553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:01.264553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:01.264553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:01.264553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:01.264553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:01.264553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:01.264553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:01.264553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 310/604] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:01.264553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069495b-04a8-47a2-9f55-9d8ed1f58998 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:01.304042 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358c8bc-7768-478a-89ed-38680d9ae1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:01.304707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358c8bc-7768-478a-89ed-38680d9ae1e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:01.304977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358c8bc-7768-478a-89ed-38680d9ae1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:01.305233 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358c8bc-7768-478a-89ed-38680d9ae1e3 None admin] PATH_INFO: `/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/d2d76fb9cbdd40f9b2a748209cee5b75/roles/5f8e9da552194885bbd7684f1a9c4f2d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:01.326644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358c8bc-7768-478a-89ed-38680d9ae1e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9e1d9331b4d4dbd3dd447937c42ad, user_id=d2d76fb9cbdd40f9b2a748209cee5b75, role_id=5f8e9da552194885bbd7684f1a9c4f2d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:01.328211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358c8bc-7768-478a-89ed-38680d9ae1e3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:01.344990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 295/605] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:09:01 2022] PUT /identity/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/d2d76fb9cbdd40f9b2a748209cee5b75/roles/5f8e9da552194885bbd7684f1a9c4f2d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:09:01.345966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358c8bc-7768-478a-89ed-38680d9ae1e3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:01.457987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:01.461039 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:01.472248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069495b-04a8-47a2-9f55-9d8ed1f58998 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:01.500677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358c8bc-7768-478a-89ed-38680d9ae1e3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:01.534281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b692110-534c-4463-9d84-94220243422f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:01.535078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b692110-534c-4463-9d84-94220243422f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:01.535360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b692110-534c-4463-9d84-94220243422f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:01.536079 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b692110-534c-4463-9d84-94220243422f None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:01.537083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b692110-534c-4463-9d84-94220243422f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:01.537395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b692110-534c-4463-9d84-94220243422f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:01.538921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b692110-534c-4463-9d84-94220243422f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:01.540772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d38ad-55b0-4d7b-b08d-85dd7fd90b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:01.541411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d38ad-55b0-4d7b-b08d-85dd7fd90b1b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:01.541693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d38ad-55b0-4d7b-b08d-85dd7fd90b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:01.541964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d38ad-55b0-4d7b-b08d-85dd7fd90b1b None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:01.542739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d38ad-55b0-4d7b-b08d-85dd7fd90b1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:01.543047 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d38ad-55b0-4d7b-b08d-85dd7fd90b1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:01.544551 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d38ad-55b0-4d7b-b08d-85dd7fd90b1b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:01.558134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 311/606] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:01.558134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b071a8-4b32-452d-91c5-20233fcee93f None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:01.558134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b071a8-4b32-452d-91c5-20233fcee93f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:01.558515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b071a8-4b32-452d-91c5-20233fcee93f None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:01.560149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 312/607] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:01.564142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 296/608] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:01.565774 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b071a8-4b32-452d-91c5-20233fcee93f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:01.567415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb11dd-eafa-45b3-87cc-cb53f15d5595 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:01.567708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb11dd-eafa-45b3-87cc-cb53f15d5595 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:01.568014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb11dd-eafa-45b3-87cc-cb53f15d5595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:01.588322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c1eb11dd-eafa-45b3-87cc-cb53f15d5595 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:01.589556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c1eb11dd-eafa-45b3-87cc-cb53f15d5595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:01.673600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 297/609] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:01.675243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb11dd-eafa-45b3-87cc-cb53f15d5595 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:01.698550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b071a8-4b32-452d-91c5-20233fcee93f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:01.737664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd400e8-80f3-4215-a19b-0d96170e7500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:01.738655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd400e8-80f3-4215-a19b-0d96170e7500 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:01.739432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd400e8-80f3-4215-a19b-0d96170e7500 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:01.739885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd400e8-80f3-4215-a19b-0d96170e7500 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:01.754242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd400e8-80f3-4215-a19b-0d96170e7500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4fd400e8-80f3-4215-a19b-0d96170e7500 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:01.761546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:01.762792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:01.763600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:01.763600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:01.763600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:01.763600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:01.763600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:01.763600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:01.763600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:01.763600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:01.763600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:01.763600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:01.763600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:01.763600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:01.766775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 313/610] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:09:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:01.767982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd400e8-80f3-4215-a19b-0d96170e7500 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:01.810935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:01.816386 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:01.889924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb11dd-eafa-45b3-87cc-cb53f15d5595 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:01.956853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd400e8-80f3-4215-a19b-0d96170e7500 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:01.960992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e169bf-8970-44ba-9030-118b34dfee81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:01.962700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e169bf-8970-44ba-9030-118b34dfee81 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:01.963489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e169bf-8970-44ba-9030-118b34dfee81 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:01.964270 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e169bf-8970-44ba-9030-118b34dfee81 None admin] PATH_INFO: `/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/e175b8de037d45d48cbddd64e552f6f2/roles/5f8e9da552194885bbd7684f1a9c4f2d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:01.987032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e169bf-8970-44ba-9030-118b34dfee81 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9e1d9331b4d4dbd3dd447937c42ad, user_id=e175b8de037d45d48cbddd64e552f6f2, role_id=5f8e9da552194885bbd7684f1a9c4f2d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:01.988885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e169bf-8970-44ba-9030-118b34dfee81 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:02.008362 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b9ad1-3fda-449d-ad02-83ee8a8d182d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:02.009740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b9ad1-3fda-449d-ad02-83ee8a8d182d None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:02.010096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b9ad1-3fda-449d-ad02-83ee8a8d182d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:02.010399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929b9ad1-3fda-449d-ad02-83ee8a8d182d None admin] PATH_INFO: `/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/e9fb56e56c7f49ce959d0bc6c515044c/roles/5f8e9da552194885bbd7684f1a9c4f2d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:02.011292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 298/611] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:09:01 2022] PUT /identity/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/e175b8de037d45d48cbddd64e552f6f2/roles/5f8e9da552194885bbd7684f1a9c4f2d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:09:02.013331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e169bf-8970-44ba-9030-118b34dfee81 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:02.047598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b9ad1-3fda-449d-ad02-83ee8a8d182d None admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9e1d9331b4d4dbd3dd447937c42ad, user_id=e9fb56e56c7f49ce959d0bc6c515044c, role_id=5f8e9da552194885bbd7684f1a9c4f2d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:02.049171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929b9ad1-3fda-449d-ad02-83ee8a8d182d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:02.075942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 314/612] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:09:01 2022] PUT /identity/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/e9fb56e56c7f49ce959d0bc6c515044c/roles/5f8e9da552194885bbd7684f1a9c4f2d => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:09:02.094730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b9ad1-3fda-449d-ad02-83ee8a8d182d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:02.226678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e169bf-8970-44ba-9030-118b34dfee81 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:02.280633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:02.289626 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:02.293860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b8985-7fa1-428b-9c86-28066216fbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:02.293860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b8985-7fa1-428b-9c86-28066216fbee None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:02.293860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b8985-7fa1-428b-9c86-28066216fbee None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:02.294732 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b8985-7fa1-428b-9c86-28066216fbee None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:02.296646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b8985-7fa1-428b-9c86-28066216fbee None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:02.297421 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b8985-7fa1-428b-9c86-28066216fbee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:02.299064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b8985-7fa1-428b-9c86-28066216fbee None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:02.311604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 299/613] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:09:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:02.318838 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b8985-7fa1-428b-9c86-28066216fbee None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:02.371777 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929b9ad1-3fda-449d-ad02-83ee8a8d182d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:02.457765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f273b-1d78-4129-bdd2-5e4bfa8a66f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:02.458451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f273b-1d78-4129-bdd2-5e4bfa8a66f0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:02.458736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f273b-1d78-4129-bdd2-5e4bfa8a66f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:02.459007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f273b-1d78-4129-bdd2-5e4bfa8a66f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:02.487541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62f273b-1d78-4129-bdd2-5e4bfa8a66f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e62f273b-1d78-4129-bdd2-5e4bfa8a66f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:02.492437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:02.493418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:02.494277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:02.494277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:02.494277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:02.494277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:02.494277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:02.494277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:02.494277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:02.494277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:02.494277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:02.494277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:02.494277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:02.494277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:02.498990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 315/614] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:09:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:02.505783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f273b-1d78-4129-bdd2-5e4bfa8a66f0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:02.525982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b8985-7fa1-428b-9c86-28066216fbee None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:02.564863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4c8b5-73c3-49ad-91c7-94642bc21c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:02.565501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4c8b5-73c3-49ad-91c7-94642bc21c7e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:02.565780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4c8b5-73c3-49ad-91c7-94642bc21c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:02.566085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4c8b5-73c3-49ad-91c7-94642bc21c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:02.579769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4c8b5-73c3-49ad-91c7-94642bc21c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:02.581233 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b4c8b5-73c3-49ad-91c7-94642bc21c7e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-07b4c8b5-73c3-49ad-91c7-94642bc21c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:02.616333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:02.617315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:02.618258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:02.618258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:02.618258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:02.622154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 300/615] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:02.625609 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4c8b5-73c3-49ad-91c7-94642bc21c7e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:02.667816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:02.764495 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f273b-1d78-4129-bdd2-5e4bfa8a66f0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:02.809251 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b4c8b5-73c3-49ad-91c7-94642bc21c7e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:02.848072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:02.870773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65e30e-c893-4a8f-833b-c1248ccb9c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:02.872053 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65e30e-c893-4a8f-833b-c1248ccb9c86 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:02.872578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65e30e-c893-4a8f-833b-c1248ccb9c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:02.872828 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65e30e-c893-4a8f-833b-c1248ccb9c86 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:02.873524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65e30e-c893-4a8f-833b-c1248ccb9c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:02.873895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65e30e-c893-4a8f-833b-c1248ccb9c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:02.875804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65e30e-c893-4a8f-833b-c1248ccb9c86 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:02.880869 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165704e-403f-4975-be25-e734d58dda7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:02.881834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165704e-403f-4975-be25-e734d58dda7c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:02.882334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165704e-403f-4975-be25-e734d58dda7c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:02.882716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165704e-403f-4975-be25-e734d58dda7c None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:02.884359 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165704e-403f-4975-be25-e734d58dda7c None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:02.884672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165704e-403f-4975-be25-e734d58dda7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:02.886561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165704e-403f-4975-be25-e734d58dda7c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:02.903622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 301/616] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:02.909563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65e30e-c893-4a8f-833b-c1248ccb9c86 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:02.918188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 316/617] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:09:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:02.924282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165704e-403f-4975-be25-e734d58dda7c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:03.030609 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:03.045957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65e30e-c893-4a8f-833b-c1248ccb9c86 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:03.087955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a50dc-df76-42a4-8563-a3fb04147289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:03.088600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a50dc-df76-42a4-8563-a3fb04147289 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:03.088881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a50dc-df76-42a4-8563-a3fb04147289 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:03.089155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a50dc-df76-42a4-8563-a3fb04147289 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:03.102953 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a50dc-df76-42a4-8563-a3fb04147289 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:03.104433 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a50dc-df76-42a4-8563-a3fb04147289 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-137a50dc-df76-42a4-8563-a3fb04147289 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:03.132857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:03.135278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:03.139658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:03.139658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:03.139658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:03.139658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:03.139658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:03.139658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:03.139658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:03.139658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:03.139658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 16:09:03.139658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:03.139658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 302/618] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:09:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:03.145741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a50dc-df76-42a4-8563-a3fb04147289 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:03.192916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165704e-403f-4975-be25-e734d58dda7c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:03.275141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51971a7f-4e3e-41a6-b0c7-c477a46bea43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:03.275811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51971a7f-4e3e-41a6-b0c7-c477a46bea43 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:03.276087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51971a7f-4e3e-41a6-b0c7-c477a46bea43 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:03.276355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51971a7f-4e3e-41a6-b0c7-c477a46bea43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:03.301070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51971a7f-4e3e-41a6-b0c7-c477a46bea43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:03.304351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51971a7f-4e3e-41a6-b0c7-c477a46bea43 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-51971a7f-4e3e-41a6-b0c7-c477a46bea43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:03.335282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:03.337662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:03.339269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:03.339269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:03.339269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:03.347826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 317/619] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:03.348803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51971a7f-4e3e-41a6-b0c7-c477a46bea43 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:03.381248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a50dc-df76-42a4-8563-a3fb04147289 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:03.442278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:03.461507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec52a1-54cb-46c8-a2b9-2c78c045078a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:03.462722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec52a1-54cb-46c8-a2b9-2c78c045078a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:03.463001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec52a1-54cb-46c8-a2b9-2c78c045078a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:03.463275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec52a1-54cb-46c8-a2b9-2c78c045078a None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:03.464018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec52a1-54cb-46c8-a2b9-2c78c045078a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:03.464324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec52a1-54cb-46c8-a2b9-2c78c045078a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:03.465776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec52a1-54cb-46c8-a2b9-2c78c045078a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:03.485411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 303/620] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Sep 21 16:09:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:03.491351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec52a1-54cb-46c8-a2b9-2c78c045078a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:03.512892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:03.584549 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51971a7f-4e3e-41a6-b0c7-c477a46bea43 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:03.670599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf32a16-c429-46cf-8c4f-c325c44b8962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:03.673429 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf32a16-c429-46cf-8c4f-c325c44b8962 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:03.674487 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf32a16-c429-46cf-8c4f-c325c44b8962 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:03.676818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf32a16-c429-46cf-8c4f-c325c44b8962 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:03.679223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf32a16-c429-46cf-8c4f-c325c44b8962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:03.680422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf32a16-c429-46cf-8c4f-c325c44b8962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:03.688064 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf32a16-c429-46cf-8c4f-c325c44b8962 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:03.697229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 318/621] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:03.703024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf32a16-c429-46cf-8c4f-c325c44b8962 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:03.767126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec52a1-54cb-46c8-a2b9-2c78c045078a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:03.822472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:03.826918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf32a16-c429-46cf-8c4f-c325c44b8962 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:03.870526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c0af94-dac8-4e6e-b2fe-ce757380488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:03.871178 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0af94-dac8-4e6e-b2fe-ce757380488c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:03.871458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0af94-dac8-4e6e-b2fe-ce757380488c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:03.871732 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0af94-dac8-4e6e-b2fe-ce757380488c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:03.890245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9efff57-7fd8-4d61-80ac-8fe6c3f7c9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:03.891197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efff57-7fd8-4d61-80ac-8fe6c3f7c9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:03.891623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efff57-7fd8-4d61-80ac-8fe6c3f7c9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:03.891798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0af94-dac8-4e6e-b2fe-ce757380488c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:03.892181 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9efff57-7fd8-4d61-80ac-8fe6c3f7c9d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:03.893268 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c0af94-dac8-4e6e-b2fe-ce757380488c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:03.901177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efff57-7fd8-4d61-80ac-8fe6c3f7c9d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:03.902888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9efff57-7fd8-4d61-80ac-8fe6c3f7c9d5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-c9efff57-7fd8-4d61-80ac-8fe6c3f7c9d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:03.916276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:03.917885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:03.919236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:03.919236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:03.919236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:03.919236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:03.919236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:03.919236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:03.919236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:03.919236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:03.919236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:09:03.919236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:03.919236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 319/622] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Sep 21 16:09:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-08c0af94-dac8-4e6e-b2fe-ce757380488c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:03.921114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:03.922400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:03.923653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:03.923653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:03.923653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:03.923819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c2a95-46f3-4870-977d-c2f7f7de2643 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:03.923819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c2a95-46f3-4870-977d-c2f7f7de2643 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:03.923819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c2a95-46f3-4870-977d-c2f7f7de2643 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:03.929640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 320/623] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:03.929832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 304/624] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 436 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:03.931306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c2a95-46f3-4870-977d-c2f7f7de2643 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:03.933231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc25d53-ac52-4d96-81b0-481bcd514af5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:03.933988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc25d53-ac52-4d96-81b0-481bcd514af5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:03.934746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc25d53-ac52-4d96-81b0-481bcd514af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:03.962894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3bc25d53-ac52-4d96-81b0-481bcd514af5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:03.965000 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3bc25d53-ac52-4d96-81b0-481bcd514af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:04.050736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c2a95-46f3-4870-977d-c2f7f7de2643 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:04.075803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:04.093286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a591a-452f-4a69-a9cb-c0d9d043b624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:04.093905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a591a-452f-4a69-a9cb-c0d9d043b624 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:04.094235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a591a-452f-4a69-a9cb-c0d9d043b624 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:04.094560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a591a-452f-4a69-a9cb-c0d9d043b624 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:04.101421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a591a-452f-4a69-a9cb-c0d9d043b624 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:04.101421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a591a-452f-4a69-a9cb-c0d9d043b624 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:04.101421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a591a-452f-4a69-a9cb-c0d9d043b624 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:04.102501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 305/625] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:04.104472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc25d53-ac52-4d96-81b0-481bcd514af5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:04.118364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 321/626] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Sep 21 16:09:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:04.123292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a591a-452f-4a69-a9cb-c0d9d043b624 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:04.166611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:04.314253 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc25d53-ac52-4d96-81b0-481bcd514af5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:04.317265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a591a-452f-4a69-a9cb-c0d9d043b624 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:04.364465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6d82e-1331-4e19-afa8-9c8568a57bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:04.365603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6d82e-1331-4e19-afa8-9c8568a57bba None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:04.366232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6d82e-1331-4e19-afa8-9c8568a57bba None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:04.366769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6d82e-1331-4e19-afa8-9c8568a57bba None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:04.367836 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6d82e-1331-4e19-afa8-9c8568a57bba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:04.368476 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6d82e-1331-4e19-afa8-9c8568a57bba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:04.370688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6d82e-1331-4e19-afa8-9c8568a57bba None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:04.384949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268fe67-ae38-4525-978f-f3f45347c965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:04.385590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268fe67-ae38-4525-978f-f3f45347c965 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:04.385869 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268fe67-ae38-4525-978f-f3f45347c965 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:04.386159 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8268fe67-ae38-4525-978f-f3f45347c965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:04.389788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 306/627] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:04.397270 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6d82e-1331-4e19-afa8-9c8568a57bba None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:04.490482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:04.577183 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268fe67-ae38-4525-978f-f3f45347c965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:04.582375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8268fe67-ae38-4525-978f-f3f45347c965 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:04.590904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6d82e-1331-4e19-afa8-9c8568a57bba None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:04.629662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:04.650653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd28c77-2092-44ed-a9fe-327ee6e08c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:04.651867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd28c77-2092-44ed-a9fe-327ee6e08c51 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:04.652406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd28c77-2092-44ed-a9fe-327ee6e08c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:04.653195 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd28c77-2092-44ed-a9fe-327ee6e08c51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:04.668913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd28c77-2092-44ed-a9fe-327ee6e08c51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:04.672495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd28c77-2092-44ed-a9fe-327ee6e08c51 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-9bd28c77-2092-44ed-a9fe-327ee6e08c51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:04.699509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:04.701372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:04.703179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:04.703179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:04.703179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:04.703179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:04.703179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:04.703179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:04.703179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:04.703179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:04.708087 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 307/628] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:09:04.715634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd28c77-2092-44ed-a9fe-327ee6e08c51 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:04.735725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 322/629] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:09:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 21 16:09:04.736623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268fe67-ae38-4525-978f-f3f45347c965 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:04.853670 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8268fe67-ae38-4525-978f-f3f45347c965 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:04.874349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:04.893738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad13786-27aa-422e-a97f-be3410766113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:04.894386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad13786-27aa-422e-a97f-be3410766113 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:04.894694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad13786-27aa-422e-a97f-be3410766113 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:04.894941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad13786-27aa-422e-a97f-be3410766113 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:04.895627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad13786-27aa-422e-a97f-be3410766113 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:04.895957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad13786-27aa-422e-a97f-be3410766113 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:04.897401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad13786-27aa-422e-a97f-be3410766113 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:04.916130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 323/630] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:04.916130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad13786-27aa-422e-a97f-be3410766113 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:04.948120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd28c77-2092-44ed-a9fe-327ee6e08c51 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:04.973844 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:05.029777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021e1b4-807b-4233-9666-b0fcb760cc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:05.031559 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021e1b4-807b-4233-9666-b0fcb760cc1d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:05.032355 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021e1b4-807b-4233-9666-b0fcb760cc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:05.032515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad13786-27aa-422e-a97f-be3410766113 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:05.033322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021e1b4-807b-4233-9666-b0fcb760cc1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:05.056186 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021e1b4-807b-4233-9666-b0fcb760cc1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b021e1b4-807b-4233-9666-b0fcb760cc1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:05.068754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:05.070633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:05.072961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:05.072961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:05.072961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:05.072961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:05.072961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:05.072961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:05.072961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:05.072961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:05.072961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:05.072961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:05.072961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:05.072961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:05.075572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce2553-3865-4084-bad8-2d8d03326165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:05.075572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce2553-3865-4084-bad8-2d8d03326165 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:05.075572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce2553-3865-4084-bad8-2d8d03326165 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:05.075572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ce2553-3865-4084-bad8-2d8d03326165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:05.079338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 308/631] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:09:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:05.081151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021e1b4-807b-4233-9666-b0fcb760cc1d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:05.092022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce2553-3865-4084-bad8-2d8d03326165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:05.093649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ce2553-3865-4084-bad8-2d8d03326165 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-d6ce2553-3865-4084-bad8-2d8d03326165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:05.109822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:05.110835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:05.111727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:05.111727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:05.111727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:05.111727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 324/632] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:05.116421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce2553-3865-4084-bad8-2d8d03326165 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:05.152234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:05.236998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ce2553-3865-4084-bad8-2d8d03326165 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:05.250401 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021e1b4-807b-4233-9666-b0fcb760cc1d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:05.276132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab7ce3-62ce-40d3-b332-e281e926f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:05.276768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab7ce3-62ce-40d3-b332-e281e926f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:05.277034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab7ce3-62ce-40d3-b332-e281e926f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:05.277373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab7ce3-62ce-40d3-b332-e281e926f6c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:05.278085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab7ce3-62ce-40d3-b332-e281e926f6c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:05.278431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab7ce3-62ce-40d3-b332-e281e926f6c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:05.279988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab7ce3-62ce-40d3-b332-e281e926f6c6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:05.289300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 325/633] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:09:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:05.291224 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad481c9-92ba-442f-89fc-b1f7646b1872 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:05.291510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad481c9-92ba-442f-89fc-b1f7646b1872 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:05.291785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad481c9-92ba-442f-89fc-b1f7646b1872 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:05.297203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 326/634] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:05.297203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad481c9-92ba-442f-89fc-b1f7646b1872 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:05.304207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edadbb-9cc3-4d4b-9fd1-e5e05cac7e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:05.305252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edadbb-9cc3-4d4b-9fd1-e5e05cac7e8e None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:05.305817 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edadbb-9cc3-4d4b-9fd1-e5e05cac7e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:05.306314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edadbb-9cc3-4d4b-9fd1-e5e05cac7e8e None admin] PATH_INFO: `/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/d71a1b8081ce490db46db95f14024ba6/roles/5f8e9da552194885bbd7684f1a9c4f2d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:05.327944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edadbb-9cc3-4d4b-9fd1-e5e05cac7e8e None admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9e1d9331b4d4dbd3dd447937c42ad, user_id=d71a1b8081ce490db46db95f14024ba6, role_id=5f8e9da552194885bbd7684f1a9c4f2d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:05.330488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edadbb-9cc3-4d4b-9fd1-e5e05cac7e8e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:05.352393 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 309/635] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:09:05 2022] PUT /identity/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/d71a1b8081ce490db46db95f14024ba6/roles/5f8e9da552194885bbd7684f1a9c4f2d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:09:05.354033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579310c0-b794-45d3-b45f-021ba14496db None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:05.354354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579310c0-b794-45d3-b45f-021ba14496db None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:05.354620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579310c0-b794-45d3-b45f-021ba14496db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:05.376214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-579310c0-b794-45d3-b45f-021ba14496db None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:05.377737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-579310c0-b794-45d3-b45f-021ba14496db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:05.398707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:05.439650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:05.460272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad481c9-92ba-442f-89fc-b1f7646b1872 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:05.473266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 310/636] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:05.475093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579310c0-b794-45d3-b45f-021ba14496db None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:05.543003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cab34-02fa-4d81-8e5e-ef8a8303b6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:05.544085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cab34-02fa-4d81-8e5e-ef8a8303b6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:05.544572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cab34-02fa-4d81-8e5e-ef8a8303b6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:05.545060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cab34-02fa-4d81-8e5e-ef8a8303b6b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:05.546317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cab34-02fa-4d81-8e5e-ef8a8303b6b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:05.546867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cab34-02fa-4d81-8e5e-ef8a8303b6b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:05.549583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cab34-02fa-4d81-8e5e-ef8a8303b6b6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:05.572920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 327/637] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:05.574546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b9639-7cd4-46fd-9fca-048af2df6308 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:05.574858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b9639-7cd4-46fd-9fca-048af2df6308 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:05.575084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b9639-7cd4-46fd-9fca-048af2df6308 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:05.578446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 328/638] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:05.582504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b9639-7cd4-46fd-9fca-048af2df6308 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:05.726441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579310c0-b794-45d3-b45f-021ba14496db None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:05.807931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ad278-84b6-4b2e-94f0-ab198bdf0a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:05.808997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ad278-84b6-4b2e-94f0-ab198bdf0a88 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:05.809473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ad278-84b6-4b2e-94f0-ab198bdf0a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:05.810072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ad278-84b6-4b2e-94f0-ab198bdf0a88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:05.815320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b9639-7cd4-46fd-9fca-048af2df6308 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:05.831587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ad278-84b6-4b2e-94f0-ab198bdf0a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:05.833036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ad278-84b6-4b2e-94f0-ab198bdf0a88 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-572ad278-84b6-4b2e-94f0-ab198bdf0a88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:05.859477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:05.861257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:05.863210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:05.863210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:05.863210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:05.863962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 311/639] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:05.870256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c3167-8a52-4f07-bb09-71474a1d8151 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:05.870256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c3167-8a52-4f07-bb09-71474a1d8151 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:05.870256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c3167-8a52-4f07-bb09-71474a1d8151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:05.877108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:05.881158 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff282b1-ebb7-4e2a-9ebb-2fea24c07d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:05.881805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff282b1-ebb7-4e2a-9ebb-2fea24c07d1c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:05.882121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff282b1-ebb7-4e2a-9ebb-2fea24c07d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:05.882398 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff282b1-ebb7-4e2a-9ebb-2fea24c07d1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:05.890717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff282b1-ebb7-4e2a-9ebb-2fea24c07d1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:05.892321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff282b1-ebb7-4e2a-9ebb-2fea24c07d1c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:05.899194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0e0c3167-8a52-4f07-bb09-71474a1d8151 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:05.900928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0e0c3167-8a52-4f07-bb09-71474a1d8151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-0ff282b1-ebb7-4e2a-9ebb-2fea24c07d1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:05.914381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:05.915382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:05.916269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:05.916269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:05.916269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:05.916269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:05.916269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:05.916269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:05.916269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:05.916269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:05.916705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 329/640] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:05.917678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff282b1-ebb7-4e2a-9ebb-2fea24c07d1c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:05.939847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:06.014978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 312/641] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:06.016336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c3167-8a52-4f07-bb09-71474a1d8151 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:06.055348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff282b1-ebb7-4e2a-9ebb-2fea24c07d1c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:06.100380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0b0f4-ba4e-4d18-9510-de3cd483fe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:06.101310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0b0f4-ba4e-4d18-9510-de3cd483fe3e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:06.101745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0b0f4-ba4e-4d18-9510-de3cd483fe3e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:06.102253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0b0f4-ba4e-4d18-9510-de3cd483fe3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:06.115299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0b0f4-ba4e-4d18-9510-de3cd483fe3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-09e0b0f4-ba4e-4d18-9510-de3cd483fe3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:06.121142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:06.122280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:06.123171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:06.123171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:06.123171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:06.123171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:06.123171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:06.123171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:06.123171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:06.123171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:06.123171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:06.123171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:06.123171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:06.123171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:06.125905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 330/642] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:09:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:06.126973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0b0f4-ba4e-4d18-9510-de3cd483fe3e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:06.218347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:06.241246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c3167-8a52-4f07-bb09-71474a1d8151 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:06.262674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0b0f4-ba4e-4d18-9510-de3cd483fe3e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:06.303702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:06.313731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dc503-f0e7-497a-a4c2-7148f8fa84f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:06.315168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dc503-f0e7-497a-a4c2-7148f8fa84f6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:06.315528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271e278-53ce-4c3f-b54b-7dabe90d51b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:06.315814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dc503-f0e7-497a-a4c2-7148f8fa84f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:06.316389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230dc503-f0e7-497a-a4c2-7148f8fa84f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:06.316747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271e278-53ce-4c3f-b54b-7dabe90d51b9 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:06.317221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271e278-53ce-4c3f-b54b-7dabe90d51b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:06.317679 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271e278-53ce-4c3f-b54b-7dabe90d51b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:06.317789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230dc503-f0e7-497a-a4c2-7148f8fa84f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:06.318233 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230dc503-f0e7-497a-a4c2-7148f8fa84f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:06.318903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271e278-53ce-4c3f-b54b-7dabe90d51b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:06.319417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271e278-53ce-4c3f-b54b-7dabe90d51b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:06.321624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271e278-53ce-4c3f-b54b-7dabe90d51b9 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:06.321750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230dc503-f0e7-497a-a4c2-7148f8fa84f6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:06.337710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 331/643] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:06.339075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271e278-53ce-4c3f-b54b-7dabe90d51b9 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:06.344596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 313/644] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:06.346334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dc503-f0e7-497a-a4c2-7148f8fa84f6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:06.468468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271e278-53ce-4c3f-b54b-7dabe90d51b9 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:06.505735 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:06.514618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95407828-4ab6-4782-af3f-e559de6ec688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:06.515252 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95407828-4ab6-4782-af3f-e559de6ec688 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:06.515550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95407828-4ab6-4782-af3f-e559de6ec688 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:06.515868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95407828-4ab6-4782-af3f-e559de6ec688 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:06.516477 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95407828-4ab6-4782-af3f-e559de6ec688 None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:06.517000 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95407828-4ab6-4782-af3f-e559de6ec688 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:06.518247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95407828-4ab6-4782-af3f-e559de6ec688 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:06.535748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 332/645] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:09:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:06.535748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95407828-4ab6-4782-af3f-e559de6ec688 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:06.562711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230dc503-f0e7-497a-a4c2-7148f8fa84f6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:06.632698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a30e0-ef8a-46e0-a705-99950e28de93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:06.633772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a30e0-ef8a-46e0-a705-99950e28de93 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:06.634267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a30e0-ef8a-46e0-a705-99950e28de93 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:06.634695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a30e0-ef8a-46e0-a705-99950e28de93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:06.655971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661a30e0-ef8a-46e0-a705-99950e28de93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:06.660518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95407828-4ab6-4782-af3f-e559de6ec688 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-661a30e0-ef8a-46e0-a705-99950e28de93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:06.669218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:06.671374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:06.673177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 314/646] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:09:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:06.675848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a30e0-ef8a-46e0-a705-99950e28de93 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:06.716028 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3d7dd-c452-476f-89fa-8e88cce22391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:06.716832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d7dd-c452-476f-89fa-8e88cce22391 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:06.717245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d7dd-c452-476f-89fa-8e88cce22391 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:06.717659 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3d7dd-c452-476f-89fa-8e88cce22391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:06.719001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3d7dd-c452-476f-89fa-8e88cce22391 None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=f0acf23d224e4b16ad03cdd9bf7cae8e) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:06.719461 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3d7dd-c452-476f-89fa-8e88cce22391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:06.721222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3d7dd-c452-476f-89fa-8e88cce22391 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:06.738536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 333/647] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:09:06 2022] GET /identity/v3/role_assignments?user.id=f0acf23d224e4b16ad03cdd9bf7cae8e&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:06.739901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3d7dd-c452-476f-89fa-8e88cce22391 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:06.783589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:06.887423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a30e0-ef8a-46e0-a705-99950e28de93 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:06.888122 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:06.966915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d36d8-7297-4e73-b1c2-b8071a4f810f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:06.968814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d36d8-7297-4e73-b1c2-b8071a4f810f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:06.970254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d36d8-7297-4e73-b1c2-b8071a4f810f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:06.971179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d36d8-7297-4e73-b1c2-b8071a4f810f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:06.981940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3d7dd-c452-476f-89fa-8e88cce22391 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:06.989244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d36d8-7297-4e73-b1c2-b8071a4f810f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:06.992181 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d36d8-7297-4e73-b1c2-b8071a4f810f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-9f5d36d8-7297-4e73-b1c2-b8071a4f810f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:07.014679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:07.017258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:07.020660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:07.020660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:07.020660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:07.020660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:07.020660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:07.020660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:07.020660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:07.020660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:07.020660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 21 16:09:07.020660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:07.020660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 315/648] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:09:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:07.020660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d36d8-7297-4e73-b1c2-b8071a4f810f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:07.062211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8487bf-b8f6-43d4-8401-ba38263b9ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:07.063835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8487bf-b8f6-43d4-8401-ba38263b9ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:07.064334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8487bf-b8f6-43d4-8401-ba38263b9ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:07.064830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8487bf-b8f6-43d4-8401-ba38263b9ea8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:07.086073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8487bf-b8f6-43d4-8401-ba38263b9ea8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:07.089459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8487bf-b8f6-43d4-8401-ba38263b9ea8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-7d8487bf-b8f6-43d4-8401-ba38263b9ea8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:07.119725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:07.120802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:07.121710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:07.121710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:07.121710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:07.122559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 334/649] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:07.124126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8487bf-b8f6-43d4-8401-ba38263b9ea8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:07.245208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d36d8-7297-4e73-b1c2-b8071a4f810f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:07.268731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8487bf-b8f6-43d4-8401-ba38263b9ea8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:07.275341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:07.284757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:07.313786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b4e67-4007-432f-a5a1-fadc8c2d9fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:07.314918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b4e67-4007-432f-a5a1-fadc8c2d9fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:07.315521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b4e67-4007-432f-a5a1-fadc8c2d9fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:07.316113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b4e67-4007-432f-a5a1-fadc8c2d9fa6 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:07.317191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b4e67-4007-432f-a5a1-fadc8c2d9fa6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:07.317798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b4e67-4007-432f-a5a1-fadc8c2d9fa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:07.319833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b4e67-4007-432f-a5a1-fadc8c2d9fa6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:07.320580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e9344-1f58-4bbb-bf68-eb9c4dc65a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:07.322253 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e9344-1f58-4bbb-bf68-eb9c4dc65a08 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:07.322747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e9344-1f58-4bbb-bf68-eb9c4dc65a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:07.323538 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22e9344-1f58-4bbb-bf68-eb9c4dc65a08 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:07.330036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e9344-1f58-4bbb-bf68-eb9c4dc65a08 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:07.330662 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e9344-1f58-4bbb-bf68-eb9c4dc65a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:07.332088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22e9344-1f58-4bbb-bf68-eb9c4dc65a08 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:07.350084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 316/650] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:09:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:07.350084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e9344-1f58-4bbb-bf68-eb9c4dc65a08 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:07.359515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 335/651] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Sep 21 16:09:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:07.361494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b4e67-4007-432f-a5a1-fadc8c2d9fa6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:07.482492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22e9344-1f58-4bbb-bf68-eb9c4dc65a08 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:07.532816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad43e6b-0c46-49e6-b045-96d3f8b48689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:07.533340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad43e6b-0c46-49e6-b045-96d3f8b48689 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:07.533624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad43e6b-0c46-49e6-b045-96d3f8b48689 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:07.533895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad43e6b-0c46-49e6-b045-96d3f8b48689 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:07.534697 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad43e6b-0c46-49e6-b045-96d3f8b48689 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:07.535006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad43e6b-0c46-49e6-b045-96d3f8b48689 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:07.536401 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad43e6b-0c46-49e6-b045-96d3f8b48689 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:07.551749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 317/652] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:07.552831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad43e6b-0c46-49e6-b045-96d3f8b48689 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:07.572332 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b4e67-4007-432f-a5a1-fadc8c2d9fa6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:07.616173 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362efa4-6ad1-4f4d-beef-c1b8d98fab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:07.617207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362efa4-6ad1-4f4d-beef-c1b8d98fab83 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:07.617781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362efa4-6ad1-4f4d-beef-c1b8d98fab83 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:07.618305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362efa4-6ad1-4f4d-beef-c1b8d98fab83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:07.618917 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:07.636320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362efa4-6ad1-4f4d-beef-c1b8d98fab83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:07.638287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362efa4-6ad1-4f4d-beef-c1b8d98fab83 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-d362efa4-6ad1-4f4d-beef-c1b8d98fab83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:07.657413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:07.658485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:07.659498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:07.659498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:07.659498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:07.666533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 336/653] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:07.667823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362efa4-6ad1-4f4d-beef-c1b8d98fab83 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:07.696669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:07.771231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad43e6b-0c46-49e6-b045-96d3f8b48689 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:07.801732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362efa4-6ad1-4f4d-beef-c1b8d98fab83 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:07.847363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a17c6a-0eb5-46aa-b696-06afec955eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:07.848004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a17c6a-0eb5-46aa-b696-06afec955eb7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:07.848302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a17c6a-0eb5-46aa-b696-06afec955eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:07.848557 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a17c6a-0eb5-46aa-b696-06afec955eb7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:07.855376 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b1ad6-cbf7-4502-b499-5637ea07ec90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:07.856046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b1ad6-cbf7-4502-b499-5637ea07ec90 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:07.856340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b1ad6-cbf7-4502-b499-5637ea07ec90 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:07.856601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504b1ad6-cbf7-4502-b499-5637ea07ec90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:07.867998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a17c6a-0eb5-46aa-b696-06afec955eb7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:07.873277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a17c6a-0eb5-46aa-b696-06afec955eb7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a4a17c6a-0eb5-46aa-b696-06afec955eb7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:07.890244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:07.892081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:07.893723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:07.893723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:07.893723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:07.893723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:07.893723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:07.893723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:07.893723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:07.893723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:07.893723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 21 16:09:07.893723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:07.893723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 337/654] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:09:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:07.893723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a17c6a-0eb5-46aa-b696-06afec955eb7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:07.982727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b1ad6-cbf7-4502-b499-5637ea07ec90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:07.984380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504b1ad6-cbf7-4502-b499-5637ea07ec90 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:08.047268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:08.052608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:08.077843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a17c6a-0eb5-46aa-b696-06afec955eb7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:08.134133 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 318/655] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:09:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:09:08.139619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b1ad6-cbf7-4502-b499-5637ea07ec90 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:08.146560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f9a32-20d7-4d7d-8cc9-b00523c68b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:08.147511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f9a32-20d7-4d7d-8cc9-b00523c68b28 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:08.147990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f9a32-20d7-4d7d-8cc9-b00523c68b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:08.148601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f9a32-20d7-4d7d-8cc9-b00523c68b28 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:08.149754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f9a32-20d7-4d7d-8cc9-b00523c68b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:08.150278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f9a32-20d7-4d7d-8cc9-b00523c68b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:08.153177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f9a32-20d7-4d7d-8cc9-b00523c68b28 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:08.182140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 338/656] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:08.182140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a899-81f7-445c-9db1-66e4558b0480 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:08.182140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a899-81f7-445c-9db1-66e4558b0480 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:08.182140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18a899-81f7-445c-9db1-66e4558b0480 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:08.183766 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 339/657] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:08.185168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18a899-81f7-445c-9db1-66e4558b0480 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:08.250728 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504b1ad6-cbf7-4502-b499-5637ea07ec90 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:08.292125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0ac93-fd53-4c54-9beb-f0a5de10ffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:08.292737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0ac93-fd53-4c54-9beb-f0a5de10ffee None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:08.293027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0ac93-fd53-4c54-9beb-f0a5de10ffee None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:08.293269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0ac93-fd53-4c54-9beb-f0a5de10ffee None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:08.294014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0ac93-fd53-4c54-9beb-f0a5de10ffee None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:08.294355 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0ac93-fd53-4c54-9beb-f0a5de10ffee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:08.295901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0ac93-fd53-4c54-9beb-f0a5de10ffee None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:08.297993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:08.324601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 319/658] 10.222.0.64 () {56 vars in 1139 bytes} [Wed Sep 21 16:09:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:08.324601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0ac93-fd53-4c54-9beb-f0a5de10ffee None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:08.379568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18a899-81f7-445c-9db1-66e4558b0480 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:08.438747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c5ae5-e2fe-4c7d-97bd-b56b341a4980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:08.439855 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c5ae5-e2fe-4c7d-97bd-b56b341a4980 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:08.440341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c5ae5-e2fe-4c7d-97bd-b56b341a4980 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:08.440832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c5ae5-e2fe-4c7d-97bd-b56b341a4980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:08.458003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:08.461756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c5ae5-e2fe-4c7d-97bd-b56b341a4980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:08.464861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c5ae5-e2fe-4c7d-97bd-b56b341a4980 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:08.473224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0ac93-fd53-4c54-9beb-f0a5de10ffee None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-206c5ae5-e2fe-4c7d-97bd-b56b341a4980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:08.495186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:08.497077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:08.498995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:08.498995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:08.498995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:08.498995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 340/659] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:08.500372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0a6cc-0cbd-4201-8fe0-94bfdf5ac92a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:08.500372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0a6cc-0cbd-4201-8fe0-94bfdf5ac92a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:08.500857 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b0a6cc-0cbd-4201-8fe0-94bfdf5ac92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:08.531484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-43b0a6cc-0cbd-4201-8fe0-94bfdf5ac92a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:08.533583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-43b0a6cc-0cbd-4201-8fe0-94bfdf5ac92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:08.541875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daf569-9f64-4667-a017-5647e8e3f2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:08.543124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daf569-9f64-4667-a017-5647e8e3f2ce None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:08.543744 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daf569-9f64-4667-a017-5647e8e3f2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:08.544359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24daf569-9f64-4667-a017-5647e8e3f2ce None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:08.558530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24daf569-9f64-4667-a017-5647e8e3f2ce None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:08.561489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24daf569-9f64-4667-a017-5647e8e3f2ce None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-24daf569-9f64-4667-a017-5647e8e3f2ce None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:08.585497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:08.587226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:08.588258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:08.588258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:08.588258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:08.588258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:08.588258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:08.588258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:08.588258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:08.588258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:08.588258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 16:09:08.588258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:08.594870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 320/660] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:09:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:08.594870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daf569-9f64-4667-a017-5647e8e3f2ce None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:08.665567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 341/661] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:08.667132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0a6cc-0cbd-4201-8fe0-94bfdf5ac92a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:08.711164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:08.782307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24daf569-9f64-4667-a017-5647e8e3f2ce None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:08.843077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1150e25-6626-4b37-915b-26e19216f7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:08.843687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1150e25-6626-4b37-915b-26e19216f7dd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:08.844048 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1150e25-6626-4b37-915b-26e19216f7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:08.844396 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1150e25-6626-4b37-915b-26e19216f7dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:08.851740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b0a6cc-0cbd-4201-8fe0-94bfdf5ac92a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:08.911126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:08.933270 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1150e25-6626-4b37-915b-26e19216f7dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:08.934773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1150e25-6626-4b37-915b-26e19216f7dd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:08.941410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906e9d1-bc8e-43bc-b6cc-12e17d410485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:08.942854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906e9d1-bc8e-43bc-b6cc-12e17d410485 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:08.943550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906e9d1-bc8e-43bc-b6cc-12e17d410485 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:08.944964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906e9d1-bc8e-43bc-b6cc-12e17d410485 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:08.947597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906e9d1-bc8e-43bc-b6cc-12e17d410485 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:08.948384 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906e9d1-bc8e-43bc-b6cc-12e17d410485 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:08.952605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906e9d1-bc8e-43bc-b6cc-12e17d410485 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:08.978911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 342/662] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:08.980971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906e9d1-bc8e-43bc-b6cc-12e17d410485 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:09.004454 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:09.067704 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 321/663] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:09:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:09:09.068732 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1150e25-6626-4b37-915b-26e19216f7dd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:09.165585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906e9d1-bc8e-43bc-b6cc-12e17d410485 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:09.205710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a3fd4-6241-46d1-bb21-282050ac2b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:09.206376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a3fd4-6241-46d1-bb21-282050ac2b9e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:09.206668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a3fd4-6241-46d1-bb21-282050ac2b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:09.206937 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a3fd4-6241-46d1-bb21-282050ac2b9e None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:09.208195 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a3fd4-6241-46d1-bb21-282050ac2b9e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:09.208195 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a3fd4-6241-46d1-bb21-282050ac2b9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:09.209547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a3fd4-6241-46d1-bb21-282050ac2b9e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:09.232682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 343/664] 10.222.0.64 () {56 vars in 1137 bytes} [Wed Sep 21 16:09:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:09.238093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a3fd4-6241-46d1-bb21-282050ac2b9e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:09.275463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1150e25-6626-4b37-915b-26e19216f7dd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:09.291516 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:09.318696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e120941-9596-4153-9286-159949aec27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:09.319474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e120941-9596-4153-9286-159949aec27e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:09.319759 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e120941-9596-4153-9286-159949aec27e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:09.320034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e120941-9596-4153-9286-159949aec27e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:09.328806 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e120941-9596-4153-9286-159949aec27e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:09.330295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e120941-9596-4153-9286-159949aec27e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-5e120941-9596-4153-9286-159949aec27e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:09.343639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:09.345518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:09.347600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:09.347600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:09.347600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:09.347600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:09.347600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:09.347600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:09.347600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:09.347600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:09.349617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 322/665] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:09.350585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e120941-9596-4153-9286-159949aec27e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:09.369126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:09.409446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a3fd4-6241-46d1-bb21-282050ac2b9e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:09.457431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce55dd8-23fb-4354-a083-fbf96115d0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:09.458174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce55dd8-23fb-4354-a083-fbf96115d0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:09.458454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce55dd8-23fb-4354-a083-fbf96115d0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:09.458731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce55dd8-23fb-4354-a083-fbf96115d0c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:09.470583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce55dd8-23fb-4354-a083-fbf96115d0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:09.472062 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce55dd8-23fb-4354-a083-fbf96115d0c2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6ce55dd8-23fb-4354-a083-fbf96115d0c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:09.495189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:09.496269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:09.497198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:09.497198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:09.497198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:09.497796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 344/666] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:09.499231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce55dd8-23fb-4354-a083-fbf96115d0c2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:09.525441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e120941-9596-4153-9286-159949aec27e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:09.578268 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d007d-e5fb-4dc9-a2a3-134b2fadd3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:09.579324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d007d-e5fb-4dc9-a2a3-134b2fadd3ca None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:09.579878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d007d-e5fb-4dc9-a2a3-134b2fadd3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:09.580247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d007d-e5fb-4dc9-a2a3-134b2fadd3ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:09.595124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32d007d-e5fb-4dc9-a2a3-134b2fadd3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a32d007d-e5fb-4dc9-a2a3-134b2fadd3ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:09.601808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:09.602913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 323/667] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:09:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:09.604388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d007d-e5fb-4dc9-a2a3-134b2fadd3ca None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:09.629598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:09.643703 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce55dd8-23fb-4354-a083-fbf96115d0c2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:09.696644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b40195-f9b9-4633-807c-1c5e440fb97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:09.697427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b40195-f9b9-4633-807c-1c5e440fb97d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:09.697694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b40195-f9b9-4633-807c-1c5e440fb97d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:09.697811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b40195-f9b9-4633-807c-1c5e440fb97d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:09.717331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:09.776729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d007d-e5fb-4dc9-a2a3-134b2fadd3ca None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:09.784349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b40195-f9b9-4633-807c-1c5e440fb97d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:09.785999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b40195-f9b9-4633-807c-1c5e440fb97d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:09.833933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bb98b-6ce1-4e05-ad67-e767e7f0e216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:09.834988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bb98b-6ce1-4e05-ad67-e767e7f0e216 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:09.835481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bb98b-6ce1-4e05-ad67-e767e7f0e216 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:09.835969 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bb98b-6ce1-4e05-ad67-e767e7f0e216 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:09.837810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bb98b-6ce1-4e05-ad67-e767e7f0e216 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:09.838382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bb98b-6ce1-4e05-ad67-e767e7f0e216 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:09.839906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bb98b-6ce1-4e05-ad67-e767e7f0e216 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:09.856857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 324/668] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:09.858617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bb98b-6ce1-4e05-ad67-e767e7f0e216 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:09.888163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:09.904418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 345/669] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:09:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:09:09.905911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b40195-f9b9-4633-807c-1c5e440fb97d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:09.981189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bb98b-6ce1-4e05-ad67-e767e7f0e216 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:10.014822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:10.020646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fdc47-5764-4d0f-9919-6137b07dd5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:10.021277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fdc47-5764-4d0f-9919-6137b07dd5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:10.021548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fdc47-5764-4d0f-9919-6137b07dd5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:10.021819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fdc47-5764-4d0f-9919-6137b07dd5a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:10.022508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fdc47-5764-4d0f-9919-6137b07dd5a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:10.022812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fdc47-5764-4d0f-9919-6137b07dd5a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:10.024258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fdc47-5764-4d0f-9919-6137b07dd5a3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:10.036487 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 325/670] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:10.037734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d605e-7c63-4354-8d59-f965a56e3e2f None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:10.038032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d605e-7c63-4354-8d59-f965a56e3e2f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:10.038340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d605e-7c63-4354-8d59-f965a56e3e2f None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:10.039596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 326/671] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:10.040324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d605e-7c63-4354-8d59-f965a56e3e2f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:10.046137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b40195-f9b9-4633-807c-1c5e440fb97d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:10.085743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c16e8-b061-46a1-8b4a-2f6de6842ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:10.086637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c16e8-b061-46a1-8b4a-2f6de6842ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:10.087012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c16e8-b061-46a1-8b4a-2f6de6842ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:10.087474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c16e8-b061-46a1-8b4a-2f6de6842ba7 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:10.088596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c16e8-b061-46a1-8b4a-2f6de6842ba7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:10.089093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c16e8-b061-46a1-8b4a-2f6de6842ba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:10.091479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c16e8-b061-46a1-8b4a-2f6de6842ba7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:10.105893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 346/672] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:09:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:10.106890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c16e8-b061-46a1-8b4a-2f6de6842ba7 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:10.174937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d605e-7c63-4354-8d59-f965a56e3e2f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:10.178567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:10.215707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb697834-94f7-4338-89cf-0e3a0cd9e5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:10.216314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb697834-94f7-4338-89cf-0e3a0cd9e5b4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:10.216598 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb697834-94f7-4338-89cf-0e3a0cd9e5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:10.216870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb697834-94f7-4338-89cf-0e3a0cd9e5b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:10.217600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb697834-94f7-4338-89cf-0e3a0cd9e5b4 None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=d2d76fb9cbdd40f9b2a748209cee5b75) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:10.217910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb697834-94f7-4338-89cf-0e3a0cd9e5b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:10.219388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb697834-94f7-4338-89cf-0e3a0cd9e5b4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:10.225809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 327/673] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:09:10 2022] GET /identity/v3/role_assignments?user.id=d2d76fb9cbdd40f9b2a748209cee5b75&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:10.226579 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb697834-94f7-4338-89cf-0e3a0cd9e5b4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:10.240115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c16e8-b061-46a1-8b4a-2f6de6842ba7 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:10.301146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e59c9-e468-4778-91ad-a98984773d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:10.301816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e59c9-e468-4778-91ad-a98984773d65 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:10.302155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e59c9-e468-4778-91ad-a98984773d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:10.302401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e59c9-e468-4778-91ad-a98984773d65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:10.312486 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:10.317017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e59c9-e468-4778-91ad-a98984773d65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:10.318552 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e59c9-e468-4778-91ad-a98984773d65 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-9b5e59c9-e468-4778-91ad-a98984773d65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:10.335675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:10.336949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:10.337828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:10.337828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:10.337828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:10.338611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 347/674] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:10.340042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5c2d5-455c-4fb1-9eb4-b4656b0056ed None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:10.340325 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5c2d5-455c-4fb1-9eb4-b4656b0056ed None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:10.340599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5c2d5-455c-4fb1-9eb4-b4656b0056ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:10.363200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b5d5c2d5-455c-4fb1-9eb4-b4656b0056ed None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:10.364444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b5d5c2d5-455c-4fb1-9eb4-b4656b0056ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:10.405349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb697834-94f7-4338-89cf-0e3a0cd9e5b4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:10.455794 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c667447-822e-414e-89eb-58d9edf06050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:10.456628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c667447-822e-414e-89eb-58d9edf06050 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:10.457284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c667447-822e-414e-89eb-58d9edf06050 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:10.457566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c667447-822e-414e-89eb-58d9edf06050 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:10.467238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 348/675] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:10.467515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5c2d5-455c-4fb1-9eb4-b4656b0056ed None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:10.469174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c667447-822e-414e-89eb-58d9edf06050 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:10.472397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c667447-822e-414e-89eb-58d9edf06050 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-8c667447-822e-414e-89eb-58d9edf06050 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:10.493308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:10.494244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:10.496402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:10.496402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:10.496402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:10.496402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:10.496402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:10.496402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:10.496402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:10.496402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:10.496402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 328/676] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:10.496886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c667447-822e-414e-89eb-58d9edf06050 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:10.519319 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:10.630768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5c2d5-455c-4fb1-9eb4-b4656b0056ed None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:10.675724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a69b1-4787-4e9e-9c02-23ab134acc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:10.676931 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a69b1-4787-4e9e-9c02-23ab134acc0a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:10.677425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a69b1-4787-4e9e-9c02-23ab134acc0a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:10.677937 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a69b1-4787-4e9e-9c02-23ab134acc0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:10.684039 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c667447-822e-414e-89eb-58d9edf06050 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:10.697942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a69b1-4787-4e9e-9c02-23ab134acc0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:10.701155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a69b1-4787-4e9e-9c02-23ab134acc0a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:10.710850 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:10.739757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294591b-5672-467a-9267-bba1fe2d07b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:10.740745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294591b-5672-467a-9267-bba1fe2d07b2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:10.741222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294591b-5672-467a-9267-bba1fe2d07b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:10.741686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a294591b-5672-467a-9267-bba1fe2d07b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:10.742927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294591b-5672-467a-9267-bba1fe2d07b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:10.743525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294591b-5672-467a-9267-bba1fe2d07b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:10.745983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a294591b-5672-467a-9267-bba1fe2d07b2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ab0a69b1-4787-4e9e-9c02-23ab134acc0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:10.747087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:10.748751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:10.751407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:10.751407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:10.751407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:10.759335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 349/677] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:10.759335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a69b1-4787-4e9e-9c02-23ab134acc0a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:10.774566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 329/678] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:10.775461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294591b-5672-467a-9267-bba1fe2d07b2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:10.891629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:10.907344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a69b1-4787-4e9e-9c02-23ab134acc0a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:10.925423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a294591b-5672-467a-9267-bba1fe2d07b2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:10.964831 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f557f89-7b18-4a15-88c8-f39e025538f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:10.965851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f557f89-7b18-4a15-88c8-f39e025538f8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:10.966377 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f557f89-7b18-4a15-88c8-f39e025538f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:10.966852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f557f89-7b18-4a15-88c8-f39e025538f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:10.968098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f557f89-7b18-4a15-88c8-f39e025538f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:10.968665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cf568-a630-4b2c-be39-8740b22acd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:10.968962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f557f89-7b18-4a15-88c8-f39e025538f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:10.969262 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cf568-a630-4b2c-be39-8740b22acd08 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:10.969534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cf568-a630-4b2c-be39-8740b22acd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:10.969820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216cf568-a630-4b2c-be39-8740b22acd08 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:10.970483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cf568-a630-4b2c-be39-8740b22acd08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:10.970789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cf568-a630-4b2c-be39-8740b22acd08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:10.971023 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f557f89-7b18-4a15-88c8-f39e025538f8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:10.972157 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216cf568-a630-4b2c-be39-8740b22acd08 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:10.990604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 330/679] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:10.990604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cf568-a630-4b2c-be39-8740b22acd08 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:10.998255 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 350/680] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:10.998255 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f557f89-7b18-4a15-88c8-f39e025538f8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:11.051798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:11.147440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216cf568-a630-4b2c-be39-8740b22acd08 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:11.159609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f557f89-7b18-4a15-88c8-f39e025538f8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:11.188654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0dc8ca-9798-463c-a0e2-c75a4ea1d2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:11.189313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0dc8ca-9798-463c-a0e2-c75a4ea1d2ef None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:11.189579 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0dc8ca-9798-463c-a0e2-c75a4ea1d2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:11.189851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0dc8ca-9798-463c-a0e2-c75a4ea1d2ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:11.204126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e727a-9956-422b-8f50-0c262481a510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:11.204881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e727a-9956-422b-8f50-0c262481a510 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:11.205229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e727a-9956-422b-8f50-0c262481a510 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:11.205570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24e727a-9956-422b-8f50-0c262481a510 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:11.207361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0dc8ca-9798-463c-a0e2-c75a4ea1d2ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7a0dc8ca-9798-463c-a0e2-c75a4ea1d2ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:11.213525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:11.214799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:11.215853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:11.215853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:11.215853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:11.215853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:11.215853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:11.215853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:11.215853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:11.215853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:11.215853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:11.215853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:11.215853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:09:11.215853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:11.219898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 331/681] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:09:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:11.220096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e727a-9956-422b-8f50-0c262481a510 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:11.220186 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0dc8ca-9798-463c-a0e2-c75a4ea1d2ef None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:11.220277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24e727a-9956-422b-8f50-0c262481a510 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-f24e727a-9956-422b-8f50-0c262481a510 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:11.239047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:11.240887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:11.243631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:11.243631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:11.243631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:11.243631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:11.243631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:11.243631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:11.243631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:11.243631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:11.243631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 351/682] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:11.243631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e727a-9956-422b-8f50-0c262481a510 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:11.257835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:11.365050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24e727a-9956-422b-8f50-0c262481a510 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:11.373339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:11.383595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0dc8ca-9798-463c-a0e2-c75a4ea1d2ef None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:11.415680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4408eb-f670-47ad-b411-db9c4541c6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:11.416531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4408eb-f670-47ad-b411-db9c4541c6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:11.417000 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4408eb-f670-47ad-b411-db9c4541c6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:11.417452 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4408eb-f670-47ad-b411-db9c4541c6c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:11.418883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4408eb-f670-47ad-b411-db9c4541c6c0 None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=e175b8de037d45d48cbddd64e552f6f2) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:11.419427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4408eb-f670-47ad-b411-db9c4541c6c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:11.422080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4408eb-f670-47ad-b411-db9c4541c6c0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:11.442088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 352/683] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:09:11 2022] GET /identity/v3/role_assignments?user.id=e175b8de037d45d48cbddd64e552f6f2&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:11.442088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4408eb-f670-47ad-b411-db9c4541c6c0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:11.456179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae44a55-a0c8-4fb9-818d-6c6f117193ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:11.457276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae44a55-a0c8-4fb9-818d-6c6f117193ee None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:11.457770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae44a55-a0c8-4fb9-818d-6c6f117193ee None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:11.458349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae44a55-a0c8-4fb9-818d-6c6f117193ee None admin] PATH_INFO: `/v3/users/swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:11.474816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae44a55-a0c8-4fb9-818d-6c6f117193ee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:11.478385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae44a55-a0c8-4fb9-818d-6c6f117193ee None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7ae44a55-a0c8-4fb9-818d-6c6f117193ee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:11.503192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:11.504266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:11.505201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:11.505201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:11.505201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:11.505201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:11.505201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:11.505201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:11.505201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:11.505201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:11.505201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 16:09:11.505201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:11.510940 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 332/684] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:09:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:11.512452 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae44a55-a0c8-4fb9-818d-6c6f117193ee None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:11.551202 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:11.606910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4408eb-f670-47ad-b411-db9c4541c6c0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:11.647097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7d3f2-9118-4248-8350-59ffd1056cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:11.648171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7d3f2-9118-4248-8350-59ffd1056cdc None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:11.648707 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7d3f2-9118-4248-8350-59ffd1056cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:11.649195 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7d3f2-9118-4248-8350-59ffd1056cdc None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:11.650217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7d3f2-9118-4248-8350-59ffd1056cdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:11.650745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7d3f2-9118-4248-8350-59ffd1056cdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:11.652429 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7d3f2-9118-4248-8350-59ffd1056cdc None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:11.664115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 353/685] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:09:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:11.666164 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7d3f2-9118-4248-8350-59ffd1056cdc None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:11.752516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae44a55-a0c8-4fb9-818d-6c6f117193ee None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:11.828728 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f0ff1-d5e2-4cb3-be68-0a85acf54451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:11.829277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f0ff1-d5e2-4cb3-be68-0a85acf54451 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:11.829560 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f0ff1-d5e2-4cb3-be68-0a85acf54451 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:11.829833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f0ff1-d5e2-4cb3-be68-0a85acf54451 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:11.830581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f0ff1-d5e2-4cb3-be68-0a85acf54451 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:11.830691 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7d3f2-9118-4248-8350-59ffd1056cdc None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:11.831107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f0ff1-d5e2-4cb3-be68-0a85acf54451 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:11.832335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f0ff1-d5e2-4cb3-be68-0a85acf54451 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:11.847000 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 333/686] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:11.847951 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f0ff1-d5e2-4cb3-be68-0a85acf54451 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:11.869689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:11.902992 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3503c6d-ec71-4648-8e1b-1ce650610843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:11.904167 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3503c6d-ec71-4648-8e1b-1ce650610843 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:11.904664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3503c6d-ec71-4648-8e1b-1ce650610843 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:11.905285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3503c6d-ec71-4648-8e1b-1ce650610843 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:11.906730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3503c6d-ec71-4648-8e1b-1ce650610843 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:11.907271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3503c6d-ec71-4648-8e1b-1ce650610843 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:11.909971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3503c6d-ec71-4648-8e1b-1ce650610843 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:11.920971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:11.935065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 354/687] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Sep 21 16:09:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:11.936268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3503c6d-ec71-4648-8e1b-1ce650610843 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:11.991541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f0ff1-d5e2-4cb3-be68-0a85acf54451 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:12.060515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3503c6d-ec71-4648-8e1b-1ce650610843 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:12.060953 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17921e1c-3dfd-4f8d-8560-11a26e3e0a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:12.061965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17921e1c-3dfd-4f8d-8560-11a26e3e0a62 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:12.062518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17921e1c-3dfd-4f8d-8560-11a26e3e0a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:12.063089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17921e1c-3dfd-4f8d-8560-11a26e3e0a62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:12.082866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17921e1c-3dfd-4f8d-8560-11a26e3e0a62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:12.085746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17921e1c-3dfd-4f8d-8560-11a26e3e0a62 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:12.101085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a42c0a-3f28-4600-8cbf-327cdf64639f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:12.102038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a42c0a-3f28-4600-8cbf-327cdf64639f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:12.102493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a42c0a-3f28-4600-8cbf-327cdf64639f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:12.103027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a42c0a-3f28-4600-8cbf-327cdf64639f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:12.103919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a42c0a-3f28-4600-8cbf-327cdf64639f None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=e9fb56e56c7f49ce959d0bc6c515044c) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:12.104371 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a42c0a-3f28-4600-8cbf-327cdf64639f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:12.106590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a42c0a-3f28-4600-8cbf-327cdf64639f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:12.114411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 355/688] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:09:11 2022] GET /identity/v3/role_assignments?user.id=e9fb56e56c7f49ce959d0bc6c515044c&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:12.115533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a42c0a-3f28-4600-8cbf-327cdf64639f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-17921e1c-3dfd-4f8d-8560-11a26e3e0a62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:12.126872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:12.128108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:12.129081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:12.129081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:12.129081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:12.129081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 334/689] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:12.138333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17921e1c-3dfd-4f8d-8560-11a26e3e0a62 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:12.190448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:12.287633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:12.324718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a42c0a-3f28-4600-8cbf-327cdf64639f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:12.365802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17921e1c-3dfd-4f8d-8560-11a26e3e0a62 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:12.436739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1a69d-64e9-4449-8875-55186395d00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:12.437811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1a69d-64e9-4449-8875-55186395d00c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:12.438312 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1a69d-64e9-4449-8875-55186395d00c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:12.438813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1a69d-64e9-4449-8875-55186395d00c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:12.453403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1af9e-0e4a-437e-b7e0-df7ffa5488f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:12.454490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1af9e-0e4a-437e-b7e0-df7ffa5488f7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:12.454787 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1af9e-0e4a-437e-b7e0-df7ffa5488f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:12.455063 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1af9e-0e4a-437e-b7e0-df7ffa5488f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:12.455735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1af9e-0e4a-437e-b7e0-df7ffa5488f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:12.456045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1af9e-0e4a-437e-b7e0-df7ffa5488f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:12.457986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1af9e-0e4a-437e-b7e0-df7ffa5488f7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:12.474534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 335/690] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:12.480925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1af9e-0e4a-437e-b7e0-df7ffa5488f7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:12.541238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1a69d-64e9-4449-8875-55186395d00c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:12.542859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f1a69d-64e9-4449-8875-55186395d00c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:12.606695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:12.666486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 356/691] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:09:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:09:12.670156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1a69d-64e9-4449-8875-55186395d00c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:12.672801 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1af9e-0e4a-437e-b7e0-df7ffa5488f7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:12.724033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:12.738389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3fbdbe-3fdc-4026-bade-7a250f743f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:12.739485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fbdbe-3fdc-4026-bade-7a250f743f74 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:12.739984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fbdbe-3fdc-4026-bade-7a250f743f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:12.740473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3fbdbe-3fdc-4026-bade-7a250f743f74 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:12.754804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fbdbe-3fdc-4026-bade-7a250f743f74 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:12.757661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3fbdbe-3fdc-4026-bade-7a250f743f74 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7a3fbdbe-3fdc-4026-bade-7a250f743f74 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:12.779702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:12.780748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:12.781934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:12.781934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:12.781934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:12.781934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:12.781934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:12.781934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:12.781934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:12.781934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:12.781934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:09:12.781934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:12.781934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 336/692] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Sep 21 16:09:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:12.786518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5ef6d-2223-4c16-8e6c-92d5418904f5 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:12.786804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5ef6d-2223-4c16-8e6c-92d5418904f5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:12.787083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5ef6d-2223-4c16-8e6c-92d5418904f5 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:12.788752 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 337/693] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:12.791130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5ef6d-2223-4c16-8e6c-92d5418904f5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:12.797114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1a69d-64e9-4449-8875-55186395d00c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:12.855506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f55b28-c8a9-4df6-b791-705713ea5777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:12.856375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f55b28-c8a9-4df6-b791-705713ea5777 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:12.856540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f55b28-c8a9-4df6-b791-705713ea5777 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:12.856842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f55b28-c8a9-4df6-b791-705713ea5777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:12.871585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f55b28-c8a9-4df6-b791-705713ea5777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:12.873802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f55b28-c8a9-4df6-b791-705713ea5777 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-b0f55b28-c8a9-4df6-b791-705713ea5777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:12.900112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:12.902322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:12.904603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:12.904603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:12.904603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:12.904603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 357/694] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:09:12.905126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9feb2-26fa-44a4-8c25-f3bb54b7827b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:12.906561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9feb2-26fa-44a4-8c25-f3bb54b7827b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:12.906561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9feb2-26fa-44a4-8c25-f3bb54b7827b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:12.916248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5ef6d-2223-4c16-8e6c-92d5418904f5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:12.936972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c2a9feb2-26fa-44a4-8c25-f3bb54b7827b None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:12.939126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c2a9feb2-26fa-44a4-8c25-f3bb54b7827b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:12.943047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:12.955113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e466f-f778-4083-b697-505915e87e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:12.955726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e466f-f778-4083-b697-505915e87e63 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:12.956055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e466f-f778-4083-b697-505915e87e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:12.956331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138e466f-f778-4083-b697-505915e87e63 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:12.957072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e466f-f778-4083-b697-505915e87e63 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:12.957428 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e466f-f778-4083-b697-505915e87e63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:12.959008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138e466f-f778-4083-b697-505915e87e63 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:12.985732 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 338/695] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Sep 21 16:09:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:12.986758 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e466f-f778-4083-b697-505915e87e63 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:13.040832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:13.077617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 358/696] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:13.079580 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9feb2-26fa-44a4-8c25-f3bb54b7827b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:13.115990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138e466f-f778-4083-b697-505915e87e63 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:13.249991 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d8233-8f43-4a51-814b-fd7b7c967547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:13.251350 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d8233-8f43-4a51-814b-fd7b7c967547 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:13.252046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d8233-8f43-4a51-814b-fd7b7c967547 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:13.252720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d8233-8f43-4a51-814b-fd7b7c967547 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:13.254142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d8233-8f43-4a51-814b-fd7b7c967547 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:13.254902 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d8233-8f43-4a51-814b-fd7b7c967547 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:13.257651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d8233-8f43-4a51-814b-fd7b7c967547 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:13.309353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 339/697] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:13.316246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d8233-8f43-4a51-814b-fd7b7c967547 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:13.376116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9feb2-26fa-44a4-8c25-f3bb54b7827b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:13.442331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bf423-0f22-4603-a4b9-567b60de1794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:13.442996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bf423-0f22-4603-a4b9-567b60de1794 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:13.443274 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bf423-0f22-4603-a4b9-567b60de1794 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:13.443548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3bf423-0f22-4603-a4b9-567b60de1794 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:13.458424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:13.518259 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d8233-8f43-4a51-814b-fd7b7c967547 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:13.528731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:13.548657 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bf423-0f22-4603-a4b9-567b60de1794 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:13.551100 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3bf423-0f22-4603-a4b9-567b60de1794 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:13.569206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97eccd3-418b-4774-80db-af805ba44f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:13.570078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97eccd3-418b-4774-80db-af805ba44f13 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:13.571069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97eccd3-418b-4774-80db-af805ba44f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:13.571354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97eccd3-418b-4774-80db-af805ba44f13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:13.580517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97eccd3-418b-4774-80db-af805ba44f13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:13.581990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97eccd3-418b-4774-80db-af805ba44f13 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c97eccd3-418b-4774-80db-af805ba44f13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:13.597796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:13.598810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:13.600304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:13.600304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:13.600304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:13.600304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:13.600304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:13.600304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:13.600304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:13.600304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:13.600304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 340/698] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:13.600304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97eccd3-418b-4774-80db-af805ba44f13 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:13.685912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 359/699] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:09:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:09:13.690981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bf423-0f22-4603-a4b9-567b60de1794 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:13.740696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:13.752951 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97eccd3-418b-4774-80db-af805ba44f13 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:13.800665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd25ec-2549-4bd7-9da0-220d11ee3163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:13.801711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd25ec-2549-4bd7-9da0-220d11ee3163 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:13.802554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd25ec-2549-4bd7-9da0-220d11ee3163 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:13.803603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd25ec-2549-4bd7-9da0-220d11ee3163 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:13.813453 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3bf423-0f22-4603-a4b9-567b60de1794 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:13.819275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd25ec-2549-4bd7-9da0-220d11ee3163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:13.821176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd25ec-2549-4bd7-9da0-220d11ee3163 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-42fd25ec-2549-4bd7-9da0-220d11ee3163 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:13.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:13.844203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:13.845956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:13.845956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:13.845956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:13.854402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 341/700] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:13.854402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd25ec-2549-4bd7-9da0-220d11ee3163 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:13.869900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bba3bb-2118-4439-b6be-e7eda099b8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:13.872126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bba3bb-2118-4439-b6be-e7eda099b8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:13.872711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bba3bb-2118-4439-b6be-e7eda099b8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:13.873055 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bba3bb-2118-4439-b6be-e7eda099b8b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:13.874597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bba3bb-2118-4439-b6be-e7eda099b8b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:13.875147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bba3bb-2118-4439-b6be-e7eda099b8b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:13.877146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bba3bb-2118-4439-b6be-e7eda099b8b5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:13.882685 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:13.917166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 360/701] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:13.917166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bba3bb-2118-4439-b6be-e7eda099b8b5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:14.031152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd25ec-2549-4bd7-9da0-220d11ee3163 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:14.083627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aae83f-30be-4bed-a740-4276d7467485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:14.084199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aae83f-30be-4bed-a740-4276d7467485 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:14.084477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aae83f-30be-4bed-a740-4276d7467485 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:14.084754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aae83f-30be-4bed-a740-4276d7467485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:14.085839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:14.094958 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bba3bb-2118-4439-b6be-e7eda099b8b5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:14.102249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aae83f-30be-4bed-a740-4276d7467485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:14.103737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aae83f-30be-4bed-a740-4276d7467485 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-97aae83f-30be-4bed-a740-4276d7467485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:14.121509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:14.123058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:14.124789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:14.124789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:14.124789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:14.124789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 342/702] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:14.124789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aae83f-30be-4bed-a740-4276d7467485 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:14.166435 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb06a26-4413-4288-8688-7e31cb3e768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:14.167462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb06a26-4413-4288-8688-7e31cb3e768c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:14.167965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb06a26-4413-4288-8688-7e31cb3e768c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:14.168463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb06a26-4413-4288-8688-7e31cb3e768c None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:14.169678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb06a26-4413-4288-8688-7e31cb3e768c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:14.170270 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb06a26-4413-4288-8688-7e31cb3e768c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:14.172889 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb06a26-4413-4288-8688-7e31cb3e768c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:14.195021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 361/703] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:14.197172 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb06a26-4413-4288-8688-7e31cb3e768c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:14.222126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:14.265495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aae83f-30be-4bed-a740-4276d7467485 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:14.306740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c097d7-f595-4588-a9a1-4429a92f04b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:14.307352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c097d7-f595-4588-a9a1-4429a92f04b9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:14.307640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c097d7-f595-4588-a9a1-4429a92f04b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:14.307892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c097d7-f595-4588-a9a1-4429a92f04b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:14.308626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c097d7-f595-4588-a9a1-4429a92f04b9 None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=d71a1b8081ce490db46db95f14024ba6) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:14.308931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c097d7-f595-4588-a9a1-4429a92f04b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:14.310404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c097d7-f595-4588-a9a1-4429a92f04b9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:14.317136 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 343/704] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:09:14 2022] GET /identity/v3/role_assignments?user.id=d71a1b8081ce490db46db95f14024ba6&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:14.318452 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298377a2-70a0-4a5d-866f-d0de0febd407 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:14.318739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298377a2-70a0-4a5d-866f-d0de0febd407 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:14.319009 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298377a2-70a0-4a5d-866f-d0de0febd407 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:14.321117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 344/705] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:14.326241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298377a2-70a0-4a5d-866f-d0de0febd407 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:14.367138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb06a26-4413-4288-8688-7e31cb3e768c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:14.459519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298377a2-70a0-4a5d-866f-d0de0febd407 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:14.464599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb2688-5fcc-446a-81d7-2f2beec3d027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:14.466106 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb2688-5fcc-446a-81d7-2f2beec3d027 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:14.466865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb2688-5fcc-446a-81d7-2f2beec3d027 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:14.467611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb2688-5fcc-446a-81d7-2f2beec3d027 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:14.470183 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb2688-5fcc-446a-81d7-2f2beec3d027 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:14.471123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb2688-5fcc-446a-81d7-2f2beec3d027 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:14.476125 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb2688-5fcc-446a-81d7-2f2beec3d027 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:14.508597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 362/706] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:14.512581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e470d9-3335-4cb2-a501-d0cb6f01efc5 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:14.512581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e470d9-3335-4cb2-a501-d0cb6f01efc5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:14.514266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e470d9-3335-4cb2-a501-d0cb6f01efc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:14.519894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c351fc2-66aa-416b-aaf3-88fbaf6c01e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:14.520864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c351fc2-66aa-416b-aaf3-88fbaf6c01e4 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:14.521354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c351fc2-66aa-416b-aaf3-88fbaf6c01e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:14.521864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c351fc2-66aa-416b-aaf3-88fbaf6c01e4 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:14.522794 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c351fc2-66aa-416b-aaf3-88fbaf6c01e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:14.524568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c351fc2-66aa-416b-aaf3-88fbaf6c01e4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:14.534736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:14.547151 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f1e470d9-3335-4cb2-a501-d0cb6f01efc5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:14.548525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f1e470d9-3335-4cb2-a501-d0cb6f01efc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:14.564046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 345/707] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:09:14 2022] POST /identity/v3/users => generated 254 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:09:14.565314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c351fc2-66aa-416b-aaf3-88fbaf6c01e4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:14.575613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:14.658594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 363/708] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:14.792115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c351fc2-66aa-416b-aaf3-88fbaf6c01e4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:14.857428 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26206303-f9ae-4bf0-92fb-93f38b7151f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:14.858588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26206303-f9ae-4bf0-92fb-93f38b7151f2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:14.859095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26206303-f9ae-4bf0-92fb-93f38b7151f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:14.859674 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26206303-f9ae-4bf0-92fb-93f38b7151f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:14.872942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26206303-f9ae-4bf0-92fb-93f38b7151f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:14.875782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26206303-f9ae-4bf0-92fb-93f38b7151f2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-26206303-f9ae-4bf0-92fb-93f38b7151f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:14.899689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:14.900628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:14.901516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:14.901516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:14.901516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:14.901516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:14.901516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:14.901516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:14.901516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:14.901516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:14.902444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 346/709] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:14.909616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e470d9-3335-4cb2-a501-d0cb6f01efc5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:14.941984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26206303-f9ae-4bf0-92fb-93f38b7151f2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:15.038925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:15.073119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e470d9-3335-4cb2-a501-d0cb6f01efc5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:15.099972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26206303-f9ae-4bf0-92fb-93f38b7151f2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:15.141518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f0829-e56e-498f-becc-71654612c898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:15.142216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f0829-e56e-498f-becc-71654612c898 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:15.142549 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f0829-e56e-498f-becc-71654612c898 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:15.142769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f0829-e56e-498f-becc-71654612c898 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:15.143469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f0829-e56e-498f-becc-71654612c898 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:15.143773 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f0829-e56e-498f-becc-71654612c898 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:15.143887 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d935d-27c1-4d51-865b-4fbe6edbdeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:15.144984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d935d-27c1-4d51-865b-4fbe6edbdeec None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:15.145217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813f0829-e56e-498f-becc-71654612c898 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:15.145493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d935d-27c1-4d51-865b-4fbe6edbdeec None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:15.146123 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d935d-27c1-4d51-865b-4fbe6edbdeec None admin] PATH_INFO: `/v3/services/network` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:15.147681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d935d-27c1-4d51-865b-4fbe6edbdeec None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:15.151162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d935d-27c1-4d51-865b-4fbe6edbdeec None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:15.167467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 364/710] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-3c0d935d-27c1-4d51-865b-4fbe6edbdeec None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:15.168948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:15.169964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 16:09:15.170898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:15.171687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f43fcf-c127-4785-974c-75a42474e036 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:15.171687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f43fcf-c127-4785-974c-75a42474e036 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:15.171687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f43fcf-c127-4785-974c-75a42474e036 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:15.175763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 365/712] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:15.176038 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 347/712] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:14 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:15.179081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f43fcf-c127-4785-974c-75a42474e036 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:15.181001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d935d-27c1-4d51-865b-4fbe6edbdeec None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:15.198877 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:15.343308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d935d-27c1-4d51-865b-4fbe6edbdeec None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:15.367785 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:15.386292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f43fcf-c127-4785-974c-75a42474e036 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:15.408444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e952ba3-3838-4165-9347-603643604105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:15.409223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e952ba3-3838-4165-9347-603643604105 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:15.409612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e952ba3-3838-4165-9347-603643604105 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:15.409982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e952ba3-3838-4165-9347-603643604105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:15.410881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e952ba3-3838-4165-9347-603643604105 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=f0acf23d224e4b16ad03cdd9bf7cae8e) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:15.411388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e952ba3-3838-4165-9347-603643604105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:15.412937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e952ba3-3838-4165-9347-603643604105 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:15.420136 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 348/713] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:09:15 2022] GET /identity/v3/role_assignments?user.id=f0acf23d224e4b16ad03cdd9bf7cae8e&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=459d01a218314543a7a8329e88b78331 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:15.421560 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac232a-d4cb-42f3-87e8-7dbeb19a66ca None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:15.422015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac232a-d4cb-42f3-87e8-7dbeb19a66ca None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:15.422436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac232a-d4cb-42f3-87e8-7dbeb19a66ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:15.453776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a7ac232a-d4cb-42f3-87e8-7dbeb19a66ca None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:15.455061 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a7ac232a-d4cb-42f3-87e8-7dbeb19a66ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:15.489383 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccf4c2-e524-4dae-ab33-2f155c01ebac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:15.490028 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccf4c2-e524-4dae-ab33-2f155c01ebac None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:15.490396 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccf4c2-e524-4dae-ab33-2f155c01ebac None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:15.490680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccf4c2-e524-4dae-ab33-2f155c01ebac None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:15.493082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccf4c2-e524-4dae-ab33-2f155c01ebac None admin] RBAC: Adding query filter params (name=network) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:15.493429 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccf4c2-e524-4dae-ab33-2f155c01ebac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:15.495851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccf4c2-e524-4dae-ab33-2f155c01ebac None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:15.511856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 366/714] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:09:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:15.511856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccf4c2-e524-4dae-ab33-2f155c01ebac None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:15.682218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 349/715] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:15.732824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:15.800975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccf4c2-e524-4dae-ab33-2f155c01ebac None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:15.873382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a8b74-126a-4b22-914b-27c0a697f68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:15.874461 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a8b74-126a-4b22-914b-27c0a697f68e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:15.874939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a8b74-126a-4b22-914b-27c0a697f68e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:15.875389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a8b74-126a-4b22-914b-27c0a697f68e None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:15.876664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a8b74-126a-4b22-914b-27c0a697f68e None admin] RBAC: Adding query filter params (type=network) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:15.877238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a8b74-126a-4b22-914b-27c0a697f68e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:15.882624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a8b74-126a-4b22-914b-27c0a697f68e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:15.895103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 367/716] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:09:15 2022] GET /identity/v3/services?type=network => generated 124 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:16.262083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac232a-d4cb-42f3-87e8-7dbeb19a66ca None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:16.475942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac232a-d4cb-42f3-87e8-7dbeb19a66ca None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:16.498291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:16.574651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f0ae0-10c6-4052-8f7c-1918cb8b2cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:16.575665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f0ae0-10c6-4052-8f7c-1918cb8b2cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:16.576162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f0ae0-10c6-4052-8f7c-1918cb8b2cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:16.576743 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f0ae0-10c6-4052-8f7c-1918cb8b2cc5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:16.577895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f0ae0-10c6-4052-8f7c-1918cb8b2cc5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:16.580563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f0ae0-10c6-4052-8f7c-1918cb8b2cc5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-3f3f0ae0-10c6-4052-8f7c-1918cb8b2cc5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:16.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:16.595480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 16:09:16.596320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:16.599322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 350/717] 10.222.0.64 () {56 vars in 1129 bytes} [Wed Sep 21 16:09:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:16.612120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a8b74-126a-4b22-914b-27c0a697f68e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:16.844233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a8b74-126a-4b22-914b-27c0a697f68e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:16.910661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1652fdd-f019-4aa5-b4e6-6c540fe2700e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:16.911689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1652fdd-f019-4aa5-b4e6-6c540fe2700e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:16.912122 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1652fdd-f019-4aa5-b4e6-6c540fe2700e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:16.912548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1652fdd-f019-4aa5-b4e6-6c540fe2700e None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:16.913583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1652fdd-f019-4aa5-b4e6-6c540fe2700e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:16.914595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1652fdd-f019-4aa5-b4e6-6c540fe2700e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:16.917000 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1652fdd-f019-4aa5-b4e6-6c540fe2700e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:16.920590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:16.930890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 368/718] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Sep 21 16:09:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:16.936501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f0ae0-10c6-4052-8f7c-1918cb8b2cc5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:17.126254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f0ae0-10c6-4052-8f7c-1918cb8b2cc5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:17.217366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7c281-1b3f-4bea-bac2-cdb8f5b413dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:17.218459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7c281-1b3f-4bea-bac2-cdb8f5b413dc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:17.218960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7c281-1b3f-4bea-bac2-cdb8f5b413dc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:17.219450 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7c281-1b3f-4bea-bac2-cdb8f5b413dc None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:17.220660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7c281-1b3f-4bea-bac2-cdb8f5b413dc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:17.221198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7c281-1b3f-4bea-bac2-cdb8f5b413dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:17.223771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7c281-1b3f-4bea-bac2-cdb8f5b413dc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:17.238993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 351/719] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Sep 21 16:09:16 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:17.887420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d6f0a-1df0-4708-8e8a-93eafa5be713 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:17.887937 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d6f0a-1df0-4708-8e8a-93eafa5be713 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:17.887937 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d6f0a-1df0-4708-8e8a-93eafa5be713 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:17.889676 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 369/720] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:17.895273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83eab87-a7ce-427b-bd8f-a5241b4ff287 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:17.895803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83eab87-a7ce-427b-bd8f-a5241b4ff287 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:17.895803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83eab87-a7ce-427b-bd8f-a5241b4ff287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:17.927287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a83eab87-a7ce-427b-bd8f-a5241b4ff287 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:17.928510 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a83eab87-a7ce-427b-bd8f-a5241b4ff287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:17.969939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:18.032673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 352/721] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:18.054886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d2f9f-4549-4106-ae9b-5e3b1b42553e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:18.055390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d2f9f-4549-4106-ae9b-5e3b1b42553e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:18.055965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d2f9f-4549-4106-ae9b-5e3b1b42553e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:18.062186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 370/722] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:18.067800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3514a6-60d4-4345-9807-c528a6739a99 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:18.068389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3514a6-60d4-4345-9807-c528a6739a99 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:18.068860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3514a6-60d4-4345-9807-c528a6739a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:18.100399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fd3514a6-60d4-4345-9807-c528a6739a99 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:18.102585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fd3514a6-60d4-4345-9807-c528a6739a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:18.213231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 353/723] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:18.422355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d2f9f-4549-4106-ae9b-5e3b1b42553e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:18.569009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d2f9f-4549-4106-ae9b-5e3b1b42553e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:18.572586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3514a6-60d4-4345-9807-c528a6739a99 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:18.645907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59993a-3904-497a-ab62-9f0924784ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:18.646591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59993a-3904-497a-ab62-9f0924784ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:18.646868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59993a-3904-497a-ab62-9f0924784ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:18.647140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59993a-3904-497a-ab62-9f0924784ea8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:18.673109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db59993a-3904-497a-ab62-9f0924784ea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-db59993a-3904-497a-ab62-9f0924784ea8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:18.682375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:18.683403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:18.684229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 371/724] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:09:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:18.691387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59993a-3904-497a-ab62-9f0924784ea8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:18.769856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:18.784674 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:18.792419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3514a6-60d4-4345-9807-c528a6739a99 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:18.836268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59993a-3904-497a-ab62-9f0924784ea8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:18.839962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb8989-8f47-4a7d-96fc-82331dcff9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:18.840616 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb8989-8f47-4a7d-96fc-82331dcff9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:18.840893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb8989-8f47-4a7d-96fc-82331dcff9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:18.841162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb8989-8f47-4a7d-96fc-82331dcff9f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:18.855538 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb8989-8f47-4a7d-96fc-82331dcff9f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-18bb8989-8f47-4a7d-96fc-82331dcff9f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:18.861093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:18.862256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:18.863107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 354/725] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:09:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:09:18.864298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa08d3-655c-4765-b695-39933cb9555d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:18.864486 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa08d3-655c-4765-b695-39933cb9555d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:18.864764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa08d3-655c-4765-b695-39933cb9555d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:18.870458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 355/726] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:18.870458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa08d3-655c-4765-b695-39933cb9555d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:18.889113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60898093-2cd0-4266-83e1-93953c1009ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:18.890516 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60898093-2cd0-4266-83e1-93953c1009ea None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:18.891008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60898093-2cd0-4266-83e1-93953c1009ea None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:18.891559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60898093-2cd0-4266-83e1-93953c1009ea None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:18.892580 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60898093-2cd0-4266-83e1-93953c1009ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:18.893094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60898093-2cd0-4266-83e1-93953c1009ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:18.894714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60898093-2cd0-4266-83e1-93953c1009ea None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:18.907945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 372/727] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:09:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:18.909752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f7728-efda-4e77-9c92-3535d352d5eb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:18.910356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f7728-efda-4e77-9c92-3535d352d5eb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:18.910815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f7728-efda-4e77-9c92-3535d352d5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:18.935339 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5c5f7728-efda-4e77-9c92-3535d352d5eb None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:18.936912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5c5f7728-efda-4e77-9c92-3535d352d5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:19.006227 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa08d3-655c-4765-b695-39933cb9555d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:19.051761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed542d-50e2-4878-a7cf-a5a90c000199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:19.051761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed542d-50e2-4878-a7cf-a5a90c000199 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:19.051761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed542d-50e2-4878-a7cf-a5a90c000199 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:19.051761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed542d-50e2-4878-a7cf-a5a90c000199 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:19.052171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed542d-50e2-4878-a7cf-a5a90c000199 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:19.052171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed542d-50e2-4878-a7cf-a5a90c000199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:19.052171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed542d-50e2-4878-a7cf-a5a90c000199 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:19.052720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 373/728] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:19.058458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f7728-efda-4e77-9c92-3535d352d5eb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:19.072498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 356/729] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:09:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:19.078489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed542d-50e2-4878-a7cf-a5a90c000199 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:19.123771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:19.129388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:19.187691 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f7728-efda-4e77-9c92-3535d352d5eb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:19.265443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580e942-a2e1-4cbe-915d-8c11234dbabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:19.266107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580e942-a2e1-4cbe-915d-8c11234dbabf None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:19.266393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580e942-a2e1-4cbe-915d-8c11234dbabf None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:19.266664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580e942-a2e1-4cbe-915d-8c11234dbabf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:19.289792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580e942-a2e1-4cbe-915d-8c11234dbabf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:19.291285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580e942-a2e1-4cbe-915d-8c11234dbabf None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-7580e942-a2e1-4cbe-915d-8c11234dbabf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:19.317281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:19.318290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:19.319221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:19.319221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:19.319221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:19.319221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 374/730] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:19.322219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed542d-50e2-4878-a7cf-a5a90c000199 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:19.325235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580e942-a2e1-4cbe-915d-8c11234dbabf None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:19.400368 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16baf9bc-2682-468c-8d77-3ac742799da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:19.401017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baf9bc-2682-468c-8d77-3ac742799da8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:19.401292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baf9bc-2682-468c-8d77-3ac742799da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:19.401562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16baf9bc-2682-468c-8d77-3ac742799da8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:19.425027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16baf9bc-2682-468c-8d77-3ac742799da8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:19.426531 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16baf9bc-2682-468c-8d77-3ac742799da8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-16baf9bc-2682-468c-8d77-3ac742799da8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:19.444800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:19.445986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:19.446981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:19.446981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:19.446981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:19.446981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 357/731] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:09:19.454151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16baf9bc-2682-468c-8d77-3ac742799da8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:19.550236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580e942-a2e1-4cbe-915d-8c11234dbabf None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:19.563989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:19.622782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601d6c8-3166-4b72-b3c9-54fd2a29f8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:19.623322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601d6c8-3166-4b72-b3c9-54fd2a29f8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:19.623602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601d6c8-3166-4b72-b3c9-54fd2a29f8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:19.623874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1601d6c8-3166-4b72-b3c9-54fd2a29f8a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:19.624531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601d6c8-3166-4b72-b3c9-54fd2a29f8a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:19.624832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601d6c8-3166-4b72-b3c9-54fd2a29f8a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:19.629951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1601d6c8-3166-4b72-b3c9-54fd2a29f8a4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:19.631935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:19.656737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 375/732] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:19.658670 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601d6c8-3166-4b72-b3c9-54fd2a29f8a4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:19.680224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16baf9bc-2682-468c-8d77-3ac742799da8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:19.721054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4684b-0946-4836-bbe3-97c020bff6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:19.721667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4684b-0946-4836-bbe3-97c020bff6ea None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:19.721944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4684b-0946-4836-bbe3-97c020bff6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:19.722249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4684b-0946-4836-bbe3-97c020bff6ea None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:19.723141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4684b-0946-4836-bbe3-97c020bff6ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:19.723590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4684b-0946-4836-bbe3-97c020bff6ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:19.725475 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4684b-0946-4836-bbe3-97c020bff6ea None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:19.748555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 358/733] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:19.748555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4684b-0946-4836-bbe3-97c020bff6ea None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:19.833445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1601d6c8-3166-4b72-b3c9-54fd2a29f8a4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:19.891029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3c56c-5001-401b-868e-f9183197c483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:19.892411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3c56c-5001-401b-868e-f9183197c483 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:19.892846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3c56c-5001-401b-868e-f9183197c483 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:19.893262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3c56c-5001-401b-868e-f9183197c483 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:19.893683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4684b-0946-4836-bbe3-97c020bff6ea None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:19.908119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3c56c-5001-401b-868e-f9183197c483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-28c3c56c-5001-401b-868e-f9183197c483 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:19.913951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:19.915027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:19.915874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:19.915874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:19.915874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:19.915874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:19.915874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:19.915874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:19.915874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:19.915874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:19.915874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:19.915874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:19.915874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:09:19.915874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:19.918425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 376/734] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:09:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:19.919565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3c56c-5001-401b-868e-f9183197c483 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:19.931799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:19.938775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a43b07-2b78-4da1-9dbf-0a758d0a3520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:19.939588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a43b07-2b78-4da1-9dbf-0a758d0a3520 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:19.940020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a43b07-2b78-4da1-9dbf-0a758d0a3520 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:19.940683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a43b07-2b78-4da1-9dbf-0a758d0a3520 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:19.952162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a43b07-2b78-4da1-9dbf-0a758d0a3520 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:19.954298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a43b07-2b78-4da1-9dbf-0a758d0a3520 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:19.955814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-06a43b07-2b78-4da1-9dbf-0a758d0a3520 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:19.973402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:19.974501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:19.975513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:19.975513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:19.975513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:19.975513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:19.975513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:19.975513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:19.975513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:19.975513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:19.975513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 16:09:19.975513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:19.979999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 359/735] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:09:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:19.981156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a43b07-2b78-4da1-9dbf-0a758d0a3520 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:20.055865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3c56c-5001-401b-868e-f9183197c483 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:20.096507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f35da-405c-47a3-bfa3-acf4816f1f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:20.097471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f35da-405c-47a3-bfa3-acf4816f1f74 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:20.097979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f35da-405c-47a3-bfa3-acf4816f1f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:20.098517 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f35da-405c-47a3-bfa3-acf4816f1f74 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:20.108098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f35da-405c-47a3-bfa3-acf4816f1f74 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:20.110145 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f35da-405c-47a3-bfa3-acf4816f1f74 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e75f35da-405c-47a3-bfa3-acf4816f1f74 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:20.125038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:20.126408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:20.127451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:20.127451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:20.127451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:20.127451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:20.127451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:20.127451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:20.127451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:20.127451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:20.127451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 16:09:20.127451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:20.128001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a43b07-2b78-4da1-9dbf-0a758d0a3520 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:20.130692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 377/736] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Sep 21 16:09:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:20.131985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f35da-405c-47a3-bfa3-acf4816f1f74 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:20.207519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b71027-6c9a-4a9d-a37f-03237bfcb76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:20.208585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b71027-6c9a-4a9d-a37f-03237bfcb76d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:20.209093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b71027-6c9a-4a9d-a37f-03237bfcb76d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:20.209571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b71027-6c9a-4a9d-a37f-03237bfcb76d None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:20.211068 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b71027-6c9a-4a9d-a37f-03237bfcb76d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:20.211581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b71027-6c9a-4a9d-a37f-03237bfcb76d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:20.213146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b71027-6c9a-4a9d-a37f-03237bfcb76d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:20.230455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 360/737] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:09:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:20.231691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b71027-6c9a-4a9d-a37f-03237bfcb76d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:20.287267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:20.298545 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:20.309556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f35da-405c-47a3-bfa3-acf4816f1f74 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:20.350923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b195f65-8148-415e-b6fa-7f120d2dd0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:20.352696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195f65-8148-415e-b6fa-7f120d2dd0c6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:20.353533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195f65-8148-415e-b6fa-7f120d2dd0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:20.354029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195f65-8148-415e-b6fa-7f120d2dd0c6 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:20.355031 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b195f65-8148-415e-b6fa-7f120d2dd0c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:20.355603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b195f65-8148-415e-b6fa-7f120d2dd0c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:20.357242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b195f65-8148-415e-b6fa-7f120d2dd0c6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:20.360751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b71027-6c9a-4a9d-a37f-03237bfcb76d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:20.383667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 378/738] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Sep 21 16:09:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:20.385373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe0647-6c89-42b8-a8da-49884b1adb64 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:20.385882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe0647-6c89-42b8-a8da-49884b1adb64 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:20.386380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe0647-6c89-42b8-a8da-49884b1adb64 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:20.388476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 379/739] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:20.389833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe0647-6c89-42b8-a8da-49884b1adb64 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:20.407453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda1b48-451e-469f-bfd3-f676d28bc3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:20.408088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda1b48-451e-469f-bfd3-f676d28bc3eb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:20.408367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda1b48-451e-469f-bfd3-f676d28bc3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:20.409287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda1b48-451e-469f-bfd3-f676d28bc3eb None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:20.409451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda1b48-451e-469f-bfd3-f676d28bc3eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:20.409702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda1b48-451e-469f-bfd3-f676d28bc3eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:20.411588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda1b48-451e-469f-bfd3-f676d28bc3eb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:20.439203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 361/740] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Sep 21 16:09:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:20.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda1b48-451e-469f-bfd3-f676d28bc3eb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:20.561684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe0647-6c89-42b8-a8da-49884b1adb64 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:20.587363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda1b48-451e-469f-bfd3-f676d28bc3eb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:20.608228 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:20.626664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46761eff-a3d5-426b-8ab0-80de3cb802c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:20.627979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46761eff-a3d5-426b-8ab0-80de3cb802c1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:20.628205 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46761eff-a3d5-426b-8ab0-80de3cb802c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:20.628681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46761eff-a3d5-426b-8ab0-80de3cb802c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:20.635793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df16c98-260a-4627-a1c8-942581bff716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:20.636412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df16c98-260a-4627-a1c8-942581bff716 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:20.636687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df16c98-260a-4627-a1c8-942581bff716 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:20.636962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df16c98-260a-4627-a1c8-942581bff716 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:20.651057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46761eff-a3d5-426b-8ab0-80de3cb802c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:20.653795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46761eff-a3d5-426b-8ab0-80de3cb802c1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:20.657079 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-46761eff-a3d5-426b-8ab0-80de3cb802c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:20.683459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:20.685309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:20.695750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:20.695750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:20.695750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:20.695750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 380/741] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:20.695750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40eeda-78fe-4333-a240-34810c33d0af None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:20.695750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40eeda-78fe-4333-a240-34810c33d0af None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:20.695750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40eeda-78fe-4333-a240-34810c33d0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:20.718591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-fe40eeda-78fe-4333-a240-34810c33d0af None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:20.720055 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fe40eeda-78fe-4333-a240-34810c33d0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:20.739641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df16c98-260a-4627-a1c8-942581bff716 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:20.741014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df16c98-260a-4627-a1c8-942581bff716 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:20.813735 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 381/742] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:20.814662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40eeda-78fe-4333-a240-34810c33d0af None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:20.897554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 362/743] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:09:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:09:20.900545 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d001ba-05a0-40b5-8d26-5ce430ab1793 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:20.901061 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d001ba-05a0-40b5-8d26-5ce430ab1793 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:20.901559 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d001ba-05a0-40b5-8d26-5ce430ab1793 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:20.904532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 363/744] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:20.912655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d001ba-05a0-40b5-8d26-5ce430ab1793 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:20.934198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:21.009158 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40eeda-78fe-4333-a240-34810c33d0af None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:21.050351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:21.073583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6990c-1407-4468-9d12-92f46fc735dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:21.074629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6990c-1407-4468-9d12-92f46fc735dc None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:21.075116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6990c-1407-4468-9d12-92f46fc735dc None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:21.075556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6990c-1407-4468-9d12-92f46fc735dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:21.075922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d001ba-05a0-40b5-8d26-5ce430ab1793 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:21.131853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7d89d-9ff2-44b9-9936-5e31fe6790b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:21.133484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7d89d-9ff2-44b9-9936-5e31fe6790b0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:21.133796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7d89d-9ff2-44b9-9936-5e31fe6790b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:21.134411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7d89d-9ff2-44b9-9936-5e31fe6790b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:21.135768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7d89d-9ff2-44b9-9936-5e31fe6790b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:21.136210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7d89d-9ff2-44b9-9936-5e31fe6790b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:21.139148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7d89d-9ff2-44b9-9936-5e31fe6790b0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:21.163710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 364/745] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:21.165556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7d89d-9ff2-44b9-9936-5e31fe6790b0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:21.186168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6990c-1407-4468-9d12-92f46fc735dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:21.187649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6990c-1407-4468-9d12-92f46fc735dc None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:21.267131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:21.301013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 382/746] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:09:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 21 16:09:21.302339 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250f2a2-6882-4d25-8559-2149ccedf42d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:21.302620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250f2a2-6882-4d25-8559-2149ccedf42d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:21.302889 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250f2a2-6882-4d25-8559-2149ccedf42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:21.310535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7d89d-9ff2-44b9-9936-5e31fe6790b0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:21.323521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2250f2a2-6882-4d25-8559-2149ccedf42d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:21.324746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2250f2a2-6882-4d25-8559-2149ccedf42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:21.343387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:21.370397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a978f-d99d-45bf-84ba-9a833f81cf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:21.371415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a978f-d99d-45bf-84ba-9a833f81cf26 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:21.371915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a978f-d99d-45bf-84ba-9a833f81cf26 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:21.372404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a978f-d99d-45bf-84ba-9a833f81cf26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:21.391500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a978f-d99d-45bf-84ba-9a833f81cf26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:21.394066 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a978f-d99d-45bf-84ba-9a833f81cf26 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:21.398255 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 383/747] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 21 16:09:21.399130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250f2a2-6882-4d25-8559-2149ccedf42d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-cd8a978f-d99d-45bf-84ba-9a833f81cf26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:21.443779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:21.445709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:21.447438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:21.447438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:21.447438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:21.448976 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 365/748] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:21.450572 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a978f-d99d-45bf-84ba-9a833f81cf26 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:21.542910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250f2a2-6882-4d25-8559-2149ccedf42d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:21.587996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62176-88c4-47a4-8e08-ac2af04ccf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:21.589132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62176-88c4-47a4-8e08-ac2af04ccf1c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:21.589466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62176-88c4-47a4-8e08-ac2af04ccf1c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:21.589803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62176-88c4-47a4-8e08-ac2af04ccf1c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:21.598874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff62176-88c4-47a4-8e08-ac2af04ccf1c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:21.600362 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff62176-88c4-47a4-8e08-ac2af04ccf1c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6ff62176-88c4-47a4-8e08-ac2af04ccf1c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:21.614419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:21.616163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:21.617967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:21.617967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:21.617967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:21.617967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:21.617967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:21.617967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:21.617967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:21.617967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:21.617967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:09:21.617967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:21.617967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 384/749] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Sep 21 16:09:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:21.617967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62176-88c4-47a4-8e08-ac2af04ccf1c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:21.648734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:21.654651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a978f-d99d-45bf-84ba-9a833f81cf26 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:21.698908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:21.717478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3eb018-ae86-4678-acc2-2290e1a014d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:21.718132 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3eb018-ae86-4678-acc2-2290e1a014d5 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:21.718416 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3eb018-ae86-4678-acc2-2290e1a014d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:21.718695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3eb018-ae86-4678-acc2-2290e1a014d5 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:21.719377 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3eb018-ae86-4678-acc2-2290e1a014d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:21.720793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3eb018-ae86-4678-acc2-2290e1a014d5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:21.747633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 366/750] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:09:21 2022] POST /identity/v3/services => generated 240 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:09:21.747924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3eb018-ae86-4678-acc2-2290e1a014d5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:21.788603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62176-88c4-47a4-8e08-ac2af04ccf1c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:21.852876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c977f-316b-42fe-91e7-9159e0868083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:21.853836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c977f-316b-42fe-91e7-9159e0868083 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:21.854303 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c977f-316b-42fe-91e7-9159e0868083 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:21.854748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c977f-316b-42fe-91e7-9159e0868083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:21.873957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c977f-316b-42fe-91e7-9159e0868083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:21.876598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c977f-316b-42fe-91e7-9159e0868083 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-d18c977f-316b-42fe-91e7-9159e0868083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:21.902771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:21.904633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:21.906886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:21.906886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:21.906886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:21.906886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 385/751] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:21.906886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c977f-316b-42fe-91e7-9159e0868083 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:21.931355 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3eb018-ae86-4678-acc2-2290e1a014d5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:21.995669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ac55a-6423-48db-bb36-e531e214d711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:21.997111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ac55a-6423-48db-bb36-e531e214d711 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:21.997516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ac55a-6423-48db-bb36-e531e214d711 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:21.997958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1ac55a-6423-48db-bb36-e531e214d711 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:21.999148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ac55a-6423-48db-bb36-e531e214d711 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:21.999596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ac55a-6423-48db-bb36-e531e214d711 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:22.001585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1ac55a-6423-48db-bb36-e531e214d711 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:22.021833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 367/752] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:22.023958 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:22.024860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ac55a-6423-48db-bb36-e531e214d711 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:22.103364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c977f-316b-42fe-91e7-9159e0868083 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:22.108572 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:22.152096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760a48e-b922-4c2c-b778-0e8b32bf13a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:22.153022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760a48e-b922-4c2c-b778-0e8b32bf13a6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:22.153022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760a48e-b922-4c2c-b778-0e8b32bf13a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:22.153300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760a48e-b922-4c2c-b778-0e8b32bf13a6 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:22.153976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760a48e-b922-4c2c-b778-0e8b32bf13a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:22.154413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760a48e-b922-4c2c-b778-0e8b32bf13a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:22.155828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760a48e-b922-4c2c-b778-0e8b32bf13a6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:22.184383 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 386/753] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Sep 21 16:09:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:22.184383 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760a48e-b922-4c2c-b778-0e8b32bf13a6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:22.237703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1ac55a-6423-48db-bb36-e531e214d711 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:22.301927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b0abc-e1fe-4ab2-920b-b775b2b577a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:22.302883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760a48e-b922-4c2c-b778-0e8b32bf13a6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:22.303100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b0abc-e1fe-4ab2-920b-b775b2b577a5 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:22.303620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b0abc-e1fe-4ab2-920b-b775b2b577a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:22.304107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b0abc-e1fe-4ab2-920b-b775b2b577a5 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:22.305031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b0abc-e1fe-4ab2-920b-b775b2b577a5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:22.307619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b0abc-e1fe-4ab2-920b-b775b2b577a5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:22.331562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:22.338719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 368/754] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:09:22 2022] POST /identity/v3/services => generated 251 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:09:22.338719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b0abc-e1fe-4ab2-920b-b775b2b577a5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:22.390094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb8fec-1908-417a-914f-3bfc188d4a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:22.392320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb8fec-1908-417a-914f-3bfc188d4a42 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:22.392961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb8fec-1908-417a-914f-3bfc188d4a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:22.393538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb8fec-1908-417a-914f-3bfc188d4a42 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:22.394925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb8fec-1908-417a-914f-3bfc188d4a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:22.395561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb8fec-1908-417a-914f-3bfc188d4a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:22.398970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb8fec-1908-417a-914f-3bfc188d4a42 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:22.425049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 387/755] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:22.425049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb8fec-1908-417a-914f-3bfc188d4a42 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:22.512926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b0abc-e1fe-4ab2-920b-b775b2b577a5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:22.515076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:22.590314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348aaf3-c294-4c1f-8c78-679c738e40ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:22.591815 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348aaf3-c294-4c1f-8c78-679c738e40ba None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:22.592705 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348aaf3-c294-4c1f-8c78-679c738e40ba None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:22.593326 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348aaf3-c294-4c1f-8c78-679c738e40ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:22.606214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348aaf3-c294-4c1f-8c78-679c738e40ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:22.608593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348aaf3-c294-4c1f-8c78-679c738e40ba None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:22.613985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb8fec-1908-417a-914f-3bfc188d4a42 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-e348aaf3-c294-4c1f-8c78-679c738e40ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:22.623124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:22.624347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:22.625426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:22.625426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:22.625426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:22.625426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:22.625426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:22.625426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:22.625426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:22.625426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:22.627214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 369/756] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:22.628236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348aaf3-c294-4c1f-8c78-679c738e40ba None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:22.690305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe971f-c924-4133-af02-c6d6206671ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:22.691308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe971f-c924-4133-af02-c6d6206671ef None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:22.691765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe971f-c924-4133-af02-c6d6206671ef None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:22.692212 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe971f-c924-4133-af02-c6d6206671ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:22.707755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe971f-c924-4133-af02-c6d6206671ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:22.709544 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe971f-c924-4133-af02-c6d6206671ef None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-dffe971f-c924-4133-af02-c6d6206671ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:22.726836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:22.727852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:22.728733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:22.728733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:22.728733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:22.731277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 388/757] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:22.736884 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe971f-c924-4133-af02-c6d6206671ef None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:22.779215 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:22.842092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348aaf3-c294-4c1f-8c78-679c738e40ba None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:22.878192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe971f-c924-4133-af02-c6d6206671ef None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:22.913570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9376a-77f5-4dbc-a736-fd36321eeb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:22.914636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9376a-77f5-4dbc-a736-fd36321eeb34 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:22.915139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9376a-77f5-4dbc-a736-fd36321eeb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:22.915624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9376a-77f5-4dbc-a736-fd36321eeb34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:22.924409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9376a-77f5-4dbc-a736-fd36321eeb34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:22.926288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9376a-77f5-4dbc-a736-fd36321eeb34 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-34b9376a-77f5-4dbc-a736-fd36321eeb34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:22.939465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:22.940463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:22.941415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:22.941415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:22.941415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:22.941415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:22.941415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:22.941415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:22.941415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:22.941415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:22.949072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 370/758] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:22.950397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9376a-77f5-4dbc-a736-fd36321eeb34 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:22.951578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a8b20-ed1a-4c5e-ae78-ee87428f5308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:22.952650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a8b20-ed1a-4c5e-ae78-ee87428f5308 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:22.953134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a8b20-ed1a-4c5e-ae78-ee87428f5308 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:22.953617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a8b20-ed1a-4c5e-ae78-ee87428f5308 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:22.955481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a8b20-ed1a-4c5e-ae78-ee87428f5308 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:22.957527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a8b20-ed1a-4c5e-ae78-ee87428f5308 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:22.960159 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a8b20-ed1a-4c5e-ae78-ee87428f5308 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:22.983084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:22.994190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 389/759] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:22.995871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a8b20-ed1a-4c5e-ae78-ee87428f5308 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:23.120943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9376a-77f5-4dbc-a736-fd36321eeb34 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:23.166117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:23.170466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e5721-fc43-490d-b760-714c95cbc65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:23.171271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e5721-fc43-490d-b760-714c95cbc65e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:23.171553 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e5721-fc43-490d-b760-714c95cbc65e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:23.171827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e5721-fc43-490d-b760-714c95cbc65e None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:23.172510 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e5721-fc43-490d-b760-714c95cbc65e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:23.172816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e5721-fc43-490d-b760-714c95cbc65e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:23.175053 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e5721-fc43-490d-b760-714c95cbc65e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:23.190915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a8b20-ed1a-4c5e-ae78-ee87428f5308 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:23.193181 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 371/760] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:23.198108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e5721-fc43-490d-b760-714c95cbc65e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:23.249070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd37d9-6fbc-41f7-8331-31df6854ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:23.250630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd37d9-6fbc-41f7-8331-31df6854ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:23.251399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd37d9-6fbc-41f7-8331-31df6854ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:23.252179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd37d9-6fbc-41f7-8331-31df6854ecb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:23.253917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd37d9-6fbc-41f7-8331-31df6854ecb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:23.254872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd37d9-6fbc-41f7-8331-31df6854ecb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:23.258673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd37d9-6fbc-41f7-8331-31df6854ecb4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:23.281003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 390/761] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:23.283084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd37d9-6fbc-41f7-8331-31df6854ecb4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:23.324115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:23.351729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e5721-fc43-490d-b760-714c95cbc65e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:23.397470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8385fa0-29bf-49dc-b6dc-215f72b21372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:23.398194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8385fa0-29bf-49dc-b6dc-215f72b21372 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:23.398485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8385fa0-29bf-49dc-b6dc-215f72b21372 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:23.398750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8385fa0-29bf-49dc-b6dc-215f72b21372 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:23.399502 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8385fa0-29bf-49dc-b6dc-215f72b21372 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=d71a1b8081ce490db46db95f14024ba6) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:23.399800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8385fa0-29bf-49dc-b6dc-215f72b21372 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:23.401327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8385fa0-29bf-49dc-b6dc-215f72b21372 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:23.414079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 372/762] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:09:23 2022] GET /identity/v3/role_assignments?user.id=d71a1b8081ce490db46db95f14024ba6&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=459d01a218314543a7a8329e88b78331 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:23.414079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8385fa0-29bf-49dc-b6dc-215f72b21372 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:23.542608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8385fa0-29bf-49dc-b6dc-215f72b21372 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:23.557849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd37d9-6fbc-41f7-8331-31df6854ecb4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:23.589150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49393b-30fc-4ef2-bed0-6d62b46be716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:23.589776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49393b-30fc-4ef2-bed0-6d62b46be716 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:23.590083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49393b-30fc-4ef2-bed0-6d62b46be716 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:23.590355 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49393b-30fc-4ef2-bed0-6d62b46be716 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:23.591123 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49393b-30fc-4ef2-bed0-6d62b46be716 None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=8da8d12ac74644f280ba31da1715f603) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:23.591435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49393b-30fc-4ef2-bed0-6d62b46be716 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:23.594504 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49393b-30fc-4ef2-bed0-6d62b46be716 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:23.603837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 373/763] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:09:23 2022] GET /identity/v3/role_assignments?user.id=8da8d12ac74644f280ba31da1715f603&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:23.631043 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ee6b1-63fe-4ac1-96ea-6ae7e82174de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:23.631705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ee6b1-63fe-4ac1-96ea-6ae7e82174de None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:23.631996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ee6b1-63fe-4ac1-96ea-6ae7e82174de None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:23.632284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ee6b1-63fe-4ac1-96ea-6ae7e82174de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:23.640799 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ee6b1-63fe-4ac1-96ea-6ae7e82174de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:23.642409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ee6b1-63fe-4ac1-96ea-6ae7e82174de None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-cf0ee6b1-63fe-4ac1-96ea-6ae7e82174de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:23.656887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:23.659146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:23.662480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:23.662480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:23.662480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:23.662480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:23.662480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:23.662480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:23.662480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:23.662480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:23.662480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 391/764] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:23.668046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49393b-30fc-4ef2-bed0-6d62b46be716 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:23.713773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:23.925540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49393b-30fc-4ef2-bed0-6d62b46be716 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:23.993688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d3f47-3a77-45f1-8dfd-e2bf80b0b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:23.995242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d3f47-3a77-45f1-8dfd-e2bf80b0b95d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:23.996150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d3f47-3a77-45f1-8dfd-e2bf80b0b95d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:23.996904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d3f47-3a77-45f1-8dfd-e2bf80b0b95d None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:23.998619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d3f47-3a77-45f1-8dfd-e2bf80b0b95d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:23.999410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d3f47-3a77-45f1-8dfd-e2bf80b0b95d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:24.002514 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d3f47-3a77-45f1-8dfd-e2bf80b0b95d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:24.028663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 374/765] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:24.034483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ee6b1-63fe-4ac1-96ea-6ae7e82174de None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:24.057173 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:24.170141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ee6b1-63fe-4ac1-96ea-6ae7e82174de None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:24.208252 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ac6ab-3a1e-4d5a-80e5-e09eb89cbebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:24.208833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ac6ab-3a1e-4d5a-80e5-e09eb89cbebd None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:24.209117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ac6ab-3a1e-4d5a-80e5-e09eb89cbebd None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:24.209392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ac6ab-3a1e-4d5a-80e5-e09eb89cbebd None admin] PATH_INFO: `/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/f0acf23d224e4b16ad03cdd9bf7cae8e/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:24.211946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ac6ab-3a1e-4d5a-80e5-e09eb89cbebd None admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9e1d9331b4d4dbd3dd447937c42ad, user_id=f0acf23d224e4b16ad03cdd9bf7cae8e, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:24.213439 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ac6ab-3a1e-4d5a-80e5-e09eb89cbebd None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:24.231614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 392/766] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:09:24 2022] PUT /identity/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/f0acf23d224e4b16ad03cdd9bf7cae8e/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:09:25.673245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530f21c-29d9-441b-93b6-b72ccbbddba5 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:25.674008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530f21c-29d9-441b-93b6-b72ccbbddba5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:25.674008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0530f21c-29d9-441b-93b6-b72ccbbddba5 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:25.675467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 375/767] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:25.682086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f760a5-c258-4c2a-a4dd-c38576e3c4c3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:25.682890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f760a5-c258-4c2a-a4dd-c38576e3c4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:25.682890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f760a5-c258-4c2a-a4dd-c38576e3c4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:25.703525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-56f760a5-c258-4c2a-a4dd-c38576e3c4c3 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:25.704764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-56f760a5-c258-4c2a-a4dd-c38576e3c4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:25.852852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:25.866614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 393/768] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:09:26.420876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530f21c-29d9-441b-93b6-b72ccbbddba5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:26.483492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a41fb0-b20e-4ab9-b900-2b4c1e5fbbaf None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:26.483986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a41fb0-b20e-4ab9-b900-2b4c1e5fbbaf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:26.484456 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a41fb0-b20e-4ab9-b900-2b4c1e5fbbaf None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:26.487152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 394/769] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:26.498833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d33637-7abf-4e01-bb63-9d29c0e89b2c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:26.499093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d33637-7abf-4e01-bb63-9d29c0e89b2c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:26.499502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d33637-7abf-4e01-bb63-9d29c0e89b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:26.559661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-79d33637-7abf-4e01-bb63-9d29c0e89b2c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:26.561648 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-79d33637-7abf-4e01-bb63-9d29c0e89b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:26.614085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:26.687749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0530f21c-29d9-441b-93b6-b72ccbbddba5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:26.793286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6433e72-cc86-4997-8f66-b2fccab5fd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:26.793929 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6433e72-cc86-4997-8f66-b2fccab5fd9e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:26.794236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6433e72-cc86-4997-8f66-b2fccab5fd9e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:26.794508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6433e72-cc86-4997-8f66-b2fccab5fd9e None admin] PATH_INFO: `/v3/services/network` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:26.795105 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6433e72-cc86-4997-8f66-b2fccab5fd9e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:26.796495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6433e72-cc86-4997-8f66-b2fccab5fd9e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:26.803284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 395/770] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b6433e72-cc86-4997-8f66-b2fccab5fd9e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:26.834600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:26.836588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:26.838914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 376/771] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:26 2022] GET /identity/v3/services/network => generated 88 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:26.851787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d33637-7abf-4e01-bb63-9d29c0e89b2c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:27.037237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d33637-7abf-4e01-bb63-9d29c0e89b2c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:27.077354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:27.087905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e2086e-a63f-4e54-8b26-0f56af81b1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:27.089356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2086e-a63f-4e54-8b26-0f56af81b1cc None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:27.089639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2086e-a63f-4e54-8b26-0f56af81b1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:27.089914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e2086e-a63f-4e54-8b26-0f56af81b1cc None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:27.090620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2086e-a63f-4e54-8b26-0f56af81b1cc None admin] RBAC: Adding query filter params (name=network) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:27.090926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2086e-a63f-4e54-8b26-0f56af81b1cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:27.092365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e2086e-a63f-4e54-8b26-0f56af81b1cc None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:27.098241 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 396/772] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:09:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:27.107888 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6433e72-cc86-4997-8f66-b2fccab5fd9e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:27.277269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6433e72-cc86-4997-8f66-b2fccab5fd9e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:27.319991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c834da6-8bd3-4f0d-9187-7e88efbf616d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:27.320699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c834da6-8bd3-4f0d-9187-7e88efbf616d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:27.320975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c834da6-8bd3-4f0d-9187-7e88efbf616d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:27.323260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 397/773] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:09:27.330356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721bba8-717e-412a-8b8b-594694b17f90 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:27.330816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721bba8-717e-412a-8b8b-594694b17f90 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:27.331301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721bba8-717e-412a-8b8b-594694b17f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:27.342500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c12cf8-ee1b-4b66-824b-6ef1dfb6a82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:27.343523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c12cf8-ee1b-4b66-824b-6ef1dfb6a82f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:27.344075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c12cf8-ee1b-4b66-824b-6ef1dfb6a82f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:27.344477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c12cf8-ee1b-4b66-824b-6ef1dfb6a82f None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:27.345632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c12cf8-ee1b-4b66-824b-6ef1dfb6a82f None admin] RBAC: Adding query filter params (type=network) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:27.346181 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c12cf8-ee1b-4b66-824b-6ef1dfb6a82f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:27.348806 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c12cf8-ee1b-4b66-824b-6ef1dfb6a82f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:27.359699 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 377/774] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:09:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:27.361245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c12cf8-ee1b-4b66-824b-6ef1dfb6a82f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:27.362656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:27.383157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d721bba8-717e-412a-8b8b-594694b17f90 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:27.385062 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d721bba8-717e-412a-8b8b-594694b17f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:27.538365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 398/775] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:27.539623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721bba8-717e-412a-8b8b-594694b17f90 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:27.568979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c12cf8-ee1b-4b66-824b-6ef1dfb6a82f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:27.656648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc70b5d-6c3f-440d-b921-c70fa55e0d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:27.657917 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc70b5d-6c3f-440d-b921-c70fa55e0d41 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:27.658248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc70b5d-6c3f-440d-b921-c70fa55e0d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:27.659046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc70b5d-6c3f-440d-b921-c70fa55e0d41 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:27.660805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc70b5d-6c3f-440d-b921-c70fa55e0d41 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:27.665697 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc70b5d-6c3f-440d-b921-c70fa55e0d41 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0dc70b5d-6c3f-440d-b921-c70fa55e0d41 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:27.692144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:27.699780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:27.701249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 378/776] 10.222.0.64 () {56 vars in 1129 bytes} [Wed Sep 21 16:09:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:27.708613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783566c-33c9-4d43-8c5c-0365a6042d9d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:27.708613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783566c-33c9-4d43-8c5c-0365a6042d9d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:27.708613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783566c-33c9-4d43-8c5c-0365a6042d9d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:27.708613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 379/777] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:27.708613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020d945-ef70-438e-9179-295204f45e13 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:27.708613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020d945-ef70-438e-9179-295204f45e13 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:27.714888 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020d945-ef70-438e-9179-295204f45e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:27.736813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721bba8-717e-412a-8b8b-594694b17f90 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:27.748304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b020d945-ef70-438e-9179-295204f45e13 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:27.750814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b020d945-ef70-438e-9179-295204f45e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:27.785068 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:27.812294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3ec9d-b8c4-4e42-8a36-9e3aec0c3355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:27.814444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3ec9d-b8c4-4e42-8a36-9e3aec0c3355 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:27.815222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3ec9d-b8c4-4e42-8a36-9e3aec0c3355 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:27.816004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3ec9d-b8c4-4e42-8a36-9e3aec0c3355 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:27.817486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3ec9d-b8c4-4e42-8a36-9e3aec0c3355 None admin] RBAC: Adding query filter params (interface=public, service_id=18b7977d2a09426eb027f4c5e4934fbd, region_id=RegionOne) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:27.818229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3ec9d-b8c4-4e42-8a36-9e3aec0c3355 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:27.822386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3ec9d-b8c4-4e42-8a36-9e3aec0c3355 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:27.840484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 399/778] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Sep 21 16:09:27 2022] GET /identity/v3/endpoints?service_id=18b7977d2a09426eb027f4c5e4934fbd&interface=public®ion_id=RegionOne => generated 194 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:27.842657 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3ec9d-b8c4-4e42-8a36-9e3aec0c3355 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:27.859198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:27.873901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 380/779] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:09:27.874841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020d945-ef70-438e-9179-295204f45e13 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:28.037571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020d945-ef70-438e-9179-295204f45e13 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:28.065631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3ec9d-b8c4-4e42-8a36-9e3aec0c3355 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:28.103578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87120962-93d4-4f35-a4d7-1e8d567e3495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:28.104624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120962-93d4-4f35-a4d7-1e8d567e3495 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:28.105086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120962-93d4-4f35-a4d7-1e8d567e3495 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:28.105553 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87120962-93d4-4f35-a4d7-1e8d567e3495 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:28.106736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87120962-93d4-4f35-a4d7-1e8d567e3495 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:28.109298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87120962-93d4-4f35-a4d7-1e8d567e3495 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:28.120555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 381/780] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Sep 21 16:09:27 2022] GET /identity/v3/services => generated 772 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:28.122849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87120962-93d4-4f35-a4d7-1e8d567e3495 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:28.143265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ae2d2-5e9f-4e6e-935d-2905f5c37eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:28.145014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ae2d2-5e9f-4e6e-935d-2905f5c37eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:28.145866 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ae2d2-5e9f-4e6e-935d-2905f5c37eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:28.146609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ae2d2-5e9f-4e6e-935d-2905f5c37eb2 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:28.148682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ae2d2-5e9f-4e6e-935d-2905f5c37eb2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:28.149692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ae2d2-5e9f-4e6e-935d-2905f5c37eb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:28.152440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ae2d2-5e9f-4e6e-935d-2905f5c37eb2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:28.161846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 400/781] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Sep 21 16:09:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:28.171828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ae2d2-5e9f-4e6e-935d-2905f5c37eb2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:28.238367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:28.327378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87120962-93d4-4f35-a4d7-1e8d567e3495 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:28.361103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:28.399588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470c2b9-00ae-40ac-a526-2ef25e3eaf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:28.400559 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470c2b9-00ae-40ac-a526-2ef25e3eaf91 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:28.400995 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470c2b9-00ae-40ac-a526-2ef25e3eaf91 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:28.401431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470c2b9-00ae-40ac-a526-2ef25e3eaf91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:28.424666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470c2b9-00ae-40ac-a526-2ef25e3eaf91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1470c2b9-00ae-40ac-a526-2ef25e3eaf91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:28.435366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:28.437310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:28.438833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 382/782] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:09:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:28.440322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77202335-990a-4b14-9965-bb4fd9eacb22 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:28.440322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77202335-990a-4b14-9965-bb4fd9eacb22 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:28.440731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77202335-990a-4b14-9965-bb4fd9eacb22 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:28.443286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 383/783] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:28.451919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77202335-990a-4b14-9965-bb4fd9eacb22 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:28.488632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ae2d2-5e9f-4e6e-935d-2905f5c37eb2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:28.559814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb21a44f-5bd7-4ec3-84b2-2cdcce8d0d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:28.561060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21a44f-5bd7-4ec3-84b2-2cdcce8d0d13 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:28.561227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21a44f-5bd7-4ec3-84b2-2cdcce8d0d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:28.561598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb21a44f-5bd7-4ec3-84b2-2cdcce8d0d13 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:28.562563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21a44f-5bd7-4ec3-84b2-2cdcce8d0d13 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:28.563075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21a44f-5bd7-4ec3-84b2-2cdcce8d0d13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:28.564630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb21a44f-5bd7-4ec3-84b2-2cdcce8d0d13 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:28.571999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 401/784] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Sep 21 16:09:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:28.573736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cc29c-5e5d-4e77-9735-b93efd07a99f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:28.574215 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cc29c-5e5d-4e77-9735-b93efd07a99f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:28.574630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cc29c-5e5d-4e77-9735-b93efd07a99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:28.600937 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ae8cc29c-5e5d-4e77-9735-b93efd07a99f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:28.602369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ae8cc29c-5e5d-4e77-9735-b93efd07a99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:28.652114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77202335-990a-4b14-9965-bb4fd9eacb22 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:28.677852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:28.702822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5baa73-d636-42d1-8b9c-6d8c5b51b768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:28.703865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5baa73-d636-42d1-8b9c-6d8c5b51b768 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:28.703865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5baa73-d636-42d1-8b9c-6d8c5b51b768 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:28.704471 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5baa73-d636-42d1-8b9c-6d8c5b51b768 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:28.705417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5baa73-d636-42d1-8b9c-6d8c5b51b768 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:28.705912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5baa73-d636-42d1-8b9c-6d8c5b51b768 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:28.708245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5baa73-d636-42d1-8b9c-6d8c5b51b768 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:28.720335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 384/785] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:09:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:28.721865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5baa73-d636-42d1-8b9c-6d8c5b51b768 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:28.737095 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 402/786] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:28.739157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cc29c-5e5d-4e77-9735-b93efd07a99f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:28.821582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:28.922084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5baa73-d636-42d1-8b9c-6d8c5b51b768 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:28.931463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cc29c-5e5d-4e77-9735-b93efd07a99f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:28.975009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69516564-b1cc-41fc-9e5c-bd16e7e3c15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:28.975690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69516564-b1cc-41fc-9e5c-bd16e7e3c15b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:28.975967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69516564-b1cc-41fc-9e5c-bd16e7e3c15b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:28.976234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69516564-b1cc-41fc-9e5c-bd16e7e3c15b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:28.976951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69516564-b1cc-41fc-9e5c-bd16e7e3c15b None admin] RBAC: Adding query filter params (interface=public, service_id=4b707adfabfa4c789c8a2c84547affcc, region_id=RegionOne) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:28.977255 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69516564-b1cc-41fc-9e5c-bd16e7e3c15b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:28.978810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69516564-b1cc-41fc-9e5c-bd16e7e3c15b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:28.985779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 403/787] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Sep 21 16:09:28 2022] GET /identity/v3/endpoints?service_id=4b707adfabfa4c789c8a2c84547affcc&interface=public®ion_id=RegionOne => generated 194 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:28.987236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69516564-b1cc-41fc-9e5c-bd16e7e3c15b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:28.991180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad8e8d-3b5f-432e-a2c1-70cb31fbcbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:28.991832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad8e8d-3b5f-432e-a2c1-70cb31fbcbdc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:28.992104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad8e8d-3b5f-432e-a2c1-70cb31fbcbdc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:28.992372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ad8e8d-3b5f-432e-a2c1-70cb31fbcbdc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:29.006632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ad8e8d-3b5f-432e-a2c1-70cb31fbcbdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-34ad8e8d-3b5f-432e-a2c1-70cb31fbcbdc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:29.012151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:29.013142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:29.013998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 385/788] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:09:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:29.014745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad8e8d-3b5f-432e-a2c1-70cb31fbcbdc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:29.063192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:29.137959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ad8e8d-3b5f-432e-a2c1-70cb31fbcbdc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:29.181321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7947c-d260-4d1e-9d4d-1ef81727677f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:29.181965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7947c-d260-4d1e-9d4d-1ef81727677f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:29.182344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7947c-d260-4d1e-9d4d-1ef81727677f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:29.182688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d7947c-d260-4d1e-9d4d-1ef81727677f None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:29.183404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7947c-d260-4d1e-9d4d-1ef81727677f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:29.184957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d7947c-d260-4d1e-9d4d-1ef81727677f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:29.198582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 386/789] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Sep 21 16:09:29 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:29.198582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7947c-d260-4d1e-9d4d-1ef81727677f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:29.226506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69516564-b1cc-41fc-9e5c-bd16e7e3c15b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:29.247642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:29.291675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a93938-15ce-41a8-aff1-bb62b0a2dac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:29.292347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a93938-15ce-41a8-aff1-bb62b0a2dac0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:29.292622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a93938-15ce-41a8-aff1-bb62b0a2dac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:29.292891 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a93938-15ce-41a8-aff1-bb62b0a2dac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:29.306732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a93938-15ce-41a8-aff1-bb62b0a2dac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:29.310589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a93938-15ce-41a8-aff1-bb62b0a2dac0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-d3a93938-15ce-41a8-aff1-bb62b0a2dac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:29.345190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:29.346308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:29.347251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:29.347251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:29.347251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:29.354779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 404/790] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:29.356421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a93938-15ce-41a8-aff1-bb62b0a2dac0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:29.419735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d7947c-d260-4d1e-9d4d-1ef81727677f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:29.421686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:29.463488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f9b1a-4a3a-48fa-b95b-879449a18def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:29.464022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f9b1a-4a3a-48fa-b95b-879449a18def None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:29.464300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f9b1a-4a3a-48fa-b95b-879449a18def None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:29.464570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f9b1a-4a3a-48fa-b95b-879449a18def None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:29.465244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f9b1a-4a3a-48fa-b95b-879449a18def None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:29.465550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f9b1a-4a3a-48fa-b95b-879449a18def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:29.466958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f9b1a-4a3a-48fa-b95b-879449a18def None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:29.478384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 387/791] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:09:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:29.480833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f9b1a-4a3a-48fa-b95b-879449a18def None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:29.578643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a93938-15ce-41a8-aff1-bb62b0a2dac0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:29.631011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f9b1a-4a3a-48fa-b95b-879449a18def None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:29.661818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e60042-d57e-440e-a386-59e4adadb43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:29.662812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e60042-d57e-440e-a386-59e4adadb43b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:29.663662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e60042-d57e-440e-a386-59e4adadb43b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:29.663903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e60042-d57e-440e-a386-59e4adadb43b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:29.676839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e60042-d57e-440e-a386-59e4adadb43b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-92e60042-d57e-440e-a386-59e4adadb43b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:29.683399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:29.685148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 405/792] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:09:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:29.686744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e60042-d57e-440e-a386-59e4adadb43b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:29.691801 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372ff74-da3c-457a-bb5c-5007b3650a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:29.693251 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372ff74-da3c-457a-bb5c-5007b3650a47 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:29.694008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372ff74-da3c-457a-bb5c-5007b3650a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:29.694837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372ff74-da3c-457a-bb5c-5007b3650a47 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:29.696244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372ff74-da3c-457a-bb5c-5007b3650a47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:29.697108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372ff74-da3c-457a-bb5c-5007b3650a47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:29.700409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372ff74-da3c-457a-bb5c-5007b3650a47 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:29.711011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:29.724639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 388/793] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:29.727621 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372ff74-da3c-457a-bb5c-5007b3650a47 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:29.776422 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:29.805672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e60042-d57e-440e-a386-59e4adadb43b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:29.842363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666a70f-71bd-4269-a77b-218b7c0ef5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:29.843421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666a70f-71bd-4269-a77b-218b7c0ef5ca None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:29.843421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666a70f-71bd-4269-a77b-218b7c0ef5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:29.843670 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a666a70f-71bd-4269-a77b-218b7c0ef5ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:29.856325 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666a70f-71bd-4269-a77b-218b7c0ef5ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:29.858527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a666a70f-71bd-4269-a77b-218b7c0ef5ca None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a666a70f-71bd-4269-a77b-218b7c0ef5ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:29.873977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:29.875807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:29.877525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:29.877525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:29.877525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:29.877525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 406/794] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:29.877525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666a70f-71bd-4269-a77b-218b7c0ef5ca None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:29.928157 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372ff74-da3c-457a-bb5c-5007b3650a47 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:29.958564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:29.986087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a666a70f-71bd-4269-a77b-218b7c0ef5ca None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:29.999430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa834fd-b3eb-45f7-87d9-4be6f783f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:30.001122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa834fd-b3eb-45f7-87d9-4be6f783f7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:30.001988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa834fd-b3eb-45f7-87d9-4be6f783f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:30.003012 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa834fd-b3eb-45f7-87d9-4be6f783f7d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:30.004596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa834fd-b3eb-45f7-87d9-4be6f783f7d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:30.005418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa834fd-b3eb-45f7-87d9-4be6f783f7d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:30.008919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa834fd-b3eb-45f7-87d9-4be6f783f7d4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:30.025205 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71c3d8-5da4-42f4-9e8c-30ef5a085c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:30.025960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71c3d8-5da4-42f4-9e8c-30ef5a085c5d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:30.026285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71c3d8-5da4-42f4-9e8c-30ef5a085c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:30.026557 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71c3d8-5da4-42f4-9e8c-30ef5a085c5d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:30.027730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 389/795] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:09:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:30.029488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa834fd-b3eb-45f7-87d9-4be6f783f7d4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:30.035666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71c3d8-5da4-42f4-9e8c-30ef5a085c5d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:30.037738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71c3d8-5da4-42f4-9e8c-30ef5a085c5d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-3f71c3d8-5da4-42f4-9e8c-30ef5a085c5d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:30.050485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:30.052609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:30.054826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:30.054826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:30.054826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:30.054826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:30.054826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:30.054826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:30.054826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:30.054826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:30.054826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 16:09:30.054826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:30.066560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 407/796] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:09:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:30.066560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71c3d8-5da4-42f4-9e8c-30ef5a085c5d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:30.173848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71c3d8-5da4-42f4-9e8c-30ef5a085c5d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:30.208035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:30.214006 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b148d-9877-40b8-9a06-364d7eb8e5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:30.214635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b148d-9877-40b8-9a06-364d7eb8e5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:30.215499 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b148d-9877-40b8-9a06-364d7eb8e5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:30.216090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b148d-9877-40b8-9a06-364d7eb8e5e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:30.228653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:30.231221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b148d-9877-40b8-9a06-364d7eb8e5e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:30.233411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b148d-9877-40b8-9a06-364d7eb8e5e3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-af3b148d-9877-40b8-9a06-364d7eb8e5e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:30.250146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:30.252191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:30.254087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:30.254087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:30.254087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:30.254087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 408/797] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:30.255379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b148d-9877-40b8-9a06-364d7eb8e5e3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:30.266590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa834fd-b3eb-45f7-87d9-4be6f783f7d4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:30.335470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc894115-e813-4a64-8275-e5b80f4bc869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:30.337039 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc894115-e813-4a64-8275-e5b80f4bc869 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:30.337749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc894115-e813-4a64-8275-e5b80f4bc869 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:30.338450 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc894115-e813-4a64-8275-e5b80f4bc869 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:30.339696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc894115-e813-4a64-8275-e5b80f4bc869 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:30.340464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc894115-e813-4a64-8275-e5b80f4bc869 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:30.343249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc894115-e813-4a64-8275-e5b80f4bc869 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:30.368457 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 390/798] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:30.370629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc894115-e813-4a64-8275-e5b80f4bc869 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:30.379686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b148d-9877-40b8-9a06-364d7eb8e5e3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:30.417627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830901cb-ce54-4865-bc0d-d57e7880186a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:30.418432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830901cb-ce54-4865-bc0d-d57e7880186a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:30.418729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830901cb-ce54-4865-bc0d-d57e7880186a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:30.418949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830901cb-ce54-4865-bc0d-d57e7880186a None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:30.419809 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830901cb-ce54-4865-bc0d-d57e7880186a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:30.420189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830901cb-ce54-4865-bc0d-d57e7880186a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:30.421739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830901cb-ce54-4865-bc0d-d57e7880186a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:30.450087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 409/799] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Sep 21 16:09:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:30.450087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830901cb-ce54-4865-bc0d-d57e7880186a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:30.503147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:30.568410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830901cb-ce54-4865-bc0d-d57e7880186a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:30.612587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d740b1-08c0-48d8-b941-4e75032da0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:30.613214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d740b1-08c0-48d8-b941-4e75032da0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:30.613500 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d740b1-08c0-48d8-b941-4e75032da0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:30.613777 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d740b1-08c0-48d8-b941-4e75032da0d1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:30.619646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc894115-e813-4a64-8275-e5b80f4bc869 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:30.622837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d740b1-08c0-48d8-b941-4e75032da0d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:30.624671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d740b1-08c0-48d8-b941-4e75032da0d1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-87d740b1-08c0-48d8-b941-4e75032da0d1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:30.638548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:30.640320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:30.642206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:30.642206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:30.642206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:30.642206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:30.642206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:30.642206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:30.642206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:30.642206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:30.642206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 16:09:30.642206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:30.642206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 410/800] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Sep 21 16:09:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:30.642206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d740b1-08c0-48d8-b941-4e75032da0d1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:30.701521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021e36-d625-473d-bb87-c0d105f66432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:30.703238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021e36-d625-473d-bb87-c0d105f66432 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:30.704020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021e36-d625-473d-bb87-c0d105f66432 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:30.704801 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021e36-d625-473d-bb87-c0d105f66432 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:30.706543 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021e36-d625-473d-bb87-c0d105f66432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:30.707348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021e36-d625-473d-bb87-c0d105f66432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:30.711226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021e36-d625-473d-bb87-c0d105f66432 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:30.734095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 391/801] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:30.736790 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021e36-d625-473d-bb87-c0d105f66432 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:30.757832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d740b1-08c0-48d8-b941-4e75032da0d1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:30.760194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:30.760667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:30.801455 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f88f0b-9b25-4057-9054-0138c0402c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:30.802162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f88f0b-9b25-4057-9054-0138c0402c4d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:30.802444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f88f0b-9b25-4057-9054-0138c0402c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:30.802717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f88f0b-9b25-4057-9054-0138c0402c4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:30.816082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f88f0b-9b25-4057-9054-0138c0402c4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:30.817561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f88f0b-9b25-4057-9054-0138c0402c4d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-86f88f0b-9b25-4057-9054-0138c0402c4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:30.834485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:30.836704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:30.838814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:30.838814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:30.838814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:30.840475 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 411/802] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:30.841935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef06382-a17f-4c64-9fcd-aad0a856cc38 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:30.842250 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef06382-a17f-4c64-9fcd-aad0a856cc38 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:30.842530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef06382-a17f-4c64-9fcd-aad0a856cc38 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:30.850873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 412/803] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:30.850873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef06382-a17f-4c64-9fcd-aad0a856cc38 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:30.976339 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef06382-a17f-4c64-9fcd-aad0a856cc38 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:31.006635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021e36-d625-473d-bb87-c0d105f66432 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:31.015871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8220f-60be-4c30-b6d1-56652c0e64fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:31.016797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8220f-60be-4c30-b6d1-56652c0e64fd None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:31.017077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8220f-60be-4c30-b6d1-56652c0e64fd None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:31.017356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8220f-60be-4c30-b6d1-56652c0e64fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:31.026032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8220f-60be-4c30-b6d1-56652c0e64fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:31.027540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8220f-60be-4c30-b6d1-56652c0e64fd None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-17f8220f-60be-4c30-b6d1-56652c0e64fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:31.040253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:31.049979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:31.052198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:31.052198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:31.052198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:31.052198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:31.052198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:31.052198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:31.052198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:31.052198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:31.052198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:09:31.052198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:31.052198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 413/804] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Sep 21 16:09:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:31.052198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8220f-60be-4c30-b6d1-56652c0e64fd None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:31.063206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623bbc2-90ec-403d-a79f-5ace719d5e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:31.063914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623bbc2-90ec-403d-a79f-5ace719d5e49 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:31.064259 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623bbc2-90ec-403d-a79f-5ace719d5e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:31.064646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623bbc2-90ec-403d-a79f-5ace719d5e49 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:31.065395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623bbc2-90ec-403d-a79f-5ace719d5e49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:31.065697 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623bbc2-90ec-403d-a79f-5ace719d5e49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:31.067178 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623bbc2-90ec-403d-a79f-5ace719d5e49 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:31.074096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:31.091685 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 392/805] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Sep 21 16:09:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:31.094339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ac764-f9d0-4fe9-ad42-0cfdc8f911c2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:31.095111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ac764-f9d0-4fe9-ad42-0cfdc8f911c2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:31.095852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819ac764-f9d0-4fe9-ad42-0cfdc8f911c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:31.120684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-819ac764-f9d0-4fe9-ad42-0cfdc8f911c2 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:31.122580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-819ac764-f9d0-4fe9-ad42-0cfdc8f911c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:31.200792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:31.218105 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 393/806] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:31.218105 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ac764-f9d0-4fe9-ad42-0cfdc8f911c2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:31.308466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8220f-60be-4c30-b6d1-56652c0e64fd None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:31.359846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a21ec-cf5b-4c6c-9b0b-5ee0a13697bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:31.360470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a21ec-cf5b-4c6c-9b0b-5ee0a13697bd None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:31.360802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a21ec-cf5b-4c6c-9b0b-5ee0a13697bd None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:31.361089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a21ec-cf5b-4c6c-9b0b-5ee0a13697bd None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:31.362072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a21ec-cf5b-4c6c-9b0b-5ee0a13697bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:31.362376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a21ec-cf5b-4c6c-9b0b-5ee0a13697bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:31.364471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a21ec-cf5b-4c6c-9b0b-5ee0a13697bd None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:31.384312 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 414/807] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:31.385896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a21ec-cf5b-4c6c-9b0b-5ee0a13697bd None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:31.424456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819ac764-f9d0-4fe9-ad42-0cfdc8f911c2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:31.470669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e8fd8-b794-4ad5-af6b-41da20858ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:31.471556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e8fd8-b794-4ad5-af6b-41da20858ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:31.472014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e8fd8-b794-4ad5-af6b-41da20858ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:31.472449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e8fd8-b794-4ad5-af6b-41da20858ec7 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:31.473374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e8fd8-b794-4ad5-af6b-41da20858ec7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:31.473834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e8fd8-b794-4ad5-af6b-41da20858ec7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:31.475528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e8fd8-b794-4ad5-af6b-41da20858ec7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:31.500954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 394/808] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Sep 21 16:09:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:31.502072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e8fd8-b794-4ad5-af6b-41da20858ec7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:31.540969 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:31.570441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:31.583870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a21ec-cf5b-4c6c-9b0b-5ee0a13697bd None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:31.620196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321964cb-25af-4317-b55e-215aa0cc2fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:31.620852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321964cb-25af-4317-b55e-215aa0cc2fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:31.621150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321964cb-25af-4317-b55e-215aa0cc2fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:31.621394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321964cb-25af-4317-b55e-215aa0cc2fd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:31.633650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321964cb-25af-4317-b55e-215aa0cc2fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:31.638005 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321964cb-25af-4317-b55e-215aa0cc2fd4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:31.652336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e8fd8-b794-4ad5-af6b-41da20858ec7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-321964cb-25af-4317-b55e-215aa0cc2fd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:31.655356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:31.657063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:31.658755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:31.658755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:31.658755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:31.658755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 415/809] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:31.658755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321964cb-25af-4317-b55e-215aa0cc2fd4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:31.703924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed063ccf-2673-4766-a4cb-b671c4fdf3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:31.704727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed063ccf-2673-4766-a4cb-b671c4fdf3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:31.705900 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed063ccf-2673-4766-a4cb-b671c4fdf3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:31.706220 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed063ccf-2673-4766-a4cb-b671c4fdf3d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:31.715833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed063ccf-2673-4766-a4cb-b671c4fdf3d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:31.717719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed063ccf-2673-4766-a4cb-b671c4fdf3d2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ed063ccf-2673-4766-a4cb-b671c4fdf3d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:31.729906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:31.732567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:31.734117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:31.734117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:31.734117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:31.734117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:31.734117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:31.734117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:31.734117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:31.734117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:31.734117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 395/810] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:31.734117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5f2f1-61d9-4b86-b5b7-d90bee92b9fc None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:31.735190 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5f2f1-61d9-4b86-b5b7-d90bee92b9fc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:31.735190 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5f2f1-61d9-4b86-b5b7-d90bee92b9fc None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:31.737146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 396/811] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:31.737146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5f2f1-61d9-4b86-b5b7-d90bee92b9fc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:31.796857 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321964cb-25af-4317-b55e-215aa0cc2fd4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:31.824980 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:31.837438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd1e31-a262-4da2-ac84-2ca4dc9d8a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:31.838196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd1e31-a262-4da2-ac84-2ca4dc9d8a74 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:31.838614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd1e31-a262-4da2-ac84-2ca4dc9d8a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:31.839055 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd1e31-a262-4da2-ac84-2ca4dc9d8a74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:31.869534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5f2f1-61d9-4b86-b5b7-d90bee92b9fc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:31.889100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:31.915630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833378c6-0fae-48c3-b865-1d7b6de03574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:31.916229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833378c6-0fae-48c3-b865-1d7b6de03574 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:31.916502 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833378c6-0fae-48c3-b865-1d7b6de03574 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:31.916766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833378c6-0fae-48c3-b865-1d7b6de03574 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:31.917420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833378c6-0fae-48c3-b865-1d7b6de03574 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:31.917717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833378c6-0fae-48c3-b865-1d7b6de03574 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:31.919103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833378c6-0fae-48c3-b865-1d7b6de03574 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:31.931607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd1e31-a262-4da2-ac84-2ca4dc9d8a74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:31.933945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd1e31-a262-4da2-ac84-2ca4dc9d8a74 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:31.938163 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 397/812] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:31.938940 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833378c6-0fae-48c3-b865-1d7b6de03574 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:32.091995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 416/813] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:09:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:09:32.091995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd1e31-a262-4da2-ac84-2ca4dc9d8a74 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:32.092275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833378c6-0fae-48c3-b865-1d7b6de03574 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:32.120553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:32.135550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c20f8a-0013-44a0-8012-6d15ac19d7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:32.136210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c20f8a-0013-44a0-8012-6d15ac19d7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:32.136493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c20f8a-0013-44a0-8012-6d15ac19d7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:32.136767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c20f8a-0013-44a0-8012-6d15ac19d7c7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:32.137390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c20f8a-0013-44a0-8012-6d15ac19d7c7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:32.138860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c20f8a-0013-44a0-8012-6d15ac19d7c7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-07c20f8a-0013-44a0-8012-6d15ac19d7c7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:32.149292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:32.151455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:32.153070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 398/814] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:31 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:32.154889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b18247-2726-4420-9d54-37e2cbf78d04 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:32.154889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b18247-2726-4420-9d54-37e2cbf78d04 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:32.154889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b18247-2726-4420-9d54-37e2cbf78d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:32.173236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-01b18247-2726-4420-9d54-37e2cbf78d04 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:32.174889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-01b18247-2726-4420-9d54-37e2cbf78d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:32.211269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:32.245393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd1e31-a262-4da2-ac84-2ca4dc9d8a74 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:32.270539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 399/815] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:32.271834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b18247-2726-4420-9d54-37e2cbf78d04 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:32.300942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba72b5-3cd8-4ec4-9c02-2c2222675af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:32.301604 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba72b5-3cd8-4ec4-9c02-2c2222675af1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:32.301885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba72b5-3cd8-4ec4-9c02-2c2222675af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:32.302194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba72b5-3cd8-4ec4-9c02-2c2222675af1 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:32.302943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba72b5-3cd8-4ec4-9c02-2c2222675af1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:32.303254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba72b5-3cd8-4ec4-9c02-2c2222675af1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:32.305272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba72b5-3cd8-4ec4-9c02-2c2222675af1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:32.327939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 417/816] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:32.327939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba72b5-3cd8-4ec4-9c02-2c2222675af1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:32.452922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b18247-2726-4420-9d54-37e2cbf78d04 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:32.467523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:32.495081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba72b5-3cd8-4ec4-9c02-2c2222675af1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:32.503159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3775b-0eed-41d7-9587-6870d3234dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:32.504141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3775b-0eed-41d7-9587-6870d3234dfb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:32.504697 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3775b-0eed-41d7-9587-6870d3234dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:32.505267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3775b-0eed-41d7-9587-6870d3234dfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:32.522862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3775b-0eed-41d7-9587-6870d3234dfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:32.525651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea3775b-0eed-41d7-9587-6870d3234dfb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:32.542458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dfb0f-0d37-4a86-b32c-b7d7345be777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:32.543102 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dfb0f-0d37-4a86-b32c-b7d7345be777 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:32.543366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dfb0f-0d37-4a86-b32c-b7d7345be777 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:32.543639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475dfb0f-0d37-4a86-b32c-b7d7345be777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-8ea3775b-0eed-41d7-9587-6870d3234dfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:32.548724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:32.550618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:32.552761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:32.552761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:32.552761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:32.552761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:32.552761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:32.552761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:32.552761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:32.552761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:32.559405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 400/817] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:32.559661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dfb0f-0d37-4a86-b32c-b7d7345be777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:32.561014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3775b-0eed-41d7-9587-6870d3234dfb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:32.561189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475dfb0f-0d37-4a86-b32c-b7d7345be777 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-475dfb0f-0d37-4a86-b32c-b7d7345be777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:32.578479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:32.580732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:32.590411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:32.590411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:32.590411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:32.590411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 418/818] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:32.590411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dfb0f-0d37-4a86-b32c-b7d7345be777 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:32.614693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:32.695861 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3775b-0eed-41d7-9587-6870d3234dfb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:32.728229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475dfb0f-0d37-4a86-b32c-b7d7345be777 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:32.735747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade9022-34cb-4ae7-bd6d-dec4cccef44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:32.736452 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade9022-34cb-4ae7-bd6d-dec4cccef44f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:32.736772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade9022-34cb-4ae7-bd6d-dec4cccef44f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:32.737084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade9022-34cb-4ae7-bd6d-dec4cccef44f None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:32.737965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade9022-34cb-4ae7-bd6d-dec4cccef44f None admin] RBAC: Adding query filter params (name=network) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:32.738348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade9022-34cb-4ae7-bd6d-dec4cccef44f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:32.740022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade9022-34cb-4ae7-bd6d-dec4cccef44f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:32.745755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 401/819] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:09:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:32.747198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade9022-34cb-4ae7-bd6d-dec4cccef44f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:32.770289 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16e35e-e9fe-4a3d-b2d3-8f30c70af9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:32.770963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16e35e-e9fe-4a3d-b2d3-8f30c70af9dd None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:32.771247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16e35e-e9fe-4a3d-b2d3-8f30c70af9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:32.771522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e16e35e-e9fe-4a3d-b2d3-8f30c70af9dd None admin] PATH_INFO: `/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/d71a1b8081ce490db46db95f14024ba6/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:32.774162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e16e35e-e9fe-4a3d-b2d3-8f30c70af9dd None admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9e1d9331b4d4dbd3dd447937c42ad, user_id=d71a1b8081ce490db46db95f14024ba6, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:32.775602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e16e35e-e9fe-4a3d-b2d3-8f30c70af9dd None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:32.785590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:32.793187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 419/820] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:09:32 2022] PUT /identity/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/d71a1b8081ce490db46db95f14024ba6/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:09:32.794922 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16e35e-e9fe-4a3d-b2d3-8f30c70af9dd None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:32.891443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:32.905060 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade9022-34cb-4ae7-bd6d-dec4cccef44f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:32.946762 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e16e35e-e9fe-4a3d-b2d3-8f30c70af9dd None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:32.954363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f5385-21aa-4260-ac91-6802b7edc1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:32.955091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f5385-21aa-4260-ac91-6802b7edc1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:32.955448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f5385-21aa-4260-ac91-6802b7edc1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:32.955782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f5385-21aa-4260-ac91-6802b7edc1f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:32.956570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f5385-21aa-4260-ac91-6802b7edc1f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:32.956937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f5385-21aa-4260-ac91-6802b7edc1f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:32.958867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f5385-21aa-4260-ac91-6802b7edc1f5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:32.982945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 402/821] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:32.982945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f5385-21aa-4260-ac91-6802b7edc1f5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:33.007248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc3192-3534-4b25-b84a-7aa07ce68538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:33.007871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc3192-3534-4b25-b84a-7aa07ce68538 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:33.008146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc3192-3534-4b25-b84a-7aa07ce68538 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:33.008417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dc3192-3534-4b25-b84a-7aa07ce68538 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:33.009072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc3192-3534-4b25-b84a-7aa07ce68538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:33.009367 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc3192-3534-4b25-b84a-7aa07ce68538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:33.010778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dc3192-3534-4b25-b84a-7aa07ce68538 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:33.025125 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 420/822] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:33.025740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc3192-3534-4b25-b84a-7aa07ce68538 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:33.098649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:33.125336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f5385-21aa-4260-ac91-6802b7edc1f5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:33.169582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea0cf2-9b8e-486a-a145-aa54e2f1a41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:33.170259 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea0cf2-9b8e-486a-a145-aa54e2f1a41f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:33.170537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea0cf2-9b8e-486a-a145-aa54e2f1a41f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:33.170826 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea0cf2-9b8e-486a-a145-aa54e2f1a41f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:33.171985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dc3192-3534-4b25-b84a-7aa07ce68538 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:33.173180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea0cf2-9b8e-486a-a145-aa54e2f1a41f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:33.174609 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea0cf2-9b8e-486a-a145-aa54e2f1a41f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-24ea0cf2-9b8e-486a-a145-aa54e2f1a41f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:33.188507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:33.189645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 21 16:09:33.190575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:33.191410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 403/823] 10.222.0.64 () {56 vars in 1129 bytes} [Wed Sep 21 16:09:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:33.192332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea0cf2-9b8e-486a-a145-aa54e2f1a41f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:33.220141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:33.233416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23b96b-4034-4c14-b15d-6b161e200635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:33.234410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23b96b-4034-4c14-b15d-6b161e200635 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:33.234783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23b96b-4034-4c14-b15d-6b161e200635 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:33.235256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23b96b-4034-4c14-b15d-6b161e200635 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:33.236135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23b96b-4034-4c14-b15d-6b161e200635 None admin] RBAC: Adding query filter params (type=network) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:33.236569 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23b96b-4034-4c14-b15d-6b161e200635 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:33.238630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23b96b-4034-4c14-b15d-6b161e200635 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:33.245527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 421/824] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:09:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:33.246609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23b96b-4034-4c14-b15d-6b161e200635 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:33.336167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea0cf2-9b8e-486a-a145-aa54e2f1a41f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:33.373103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23b96b-4034-4c14-b15d-6b161e200635 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:33.392089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63040444-4d4e-4779-897c-e6a0439b0baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:33.393222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63040444-4d4e-4779-897c-e6a0439b0baf None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:33.393636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63040444-4d4e-4779-897c-e6a0439b0baf None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:33.394078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63040444-4d4e-4779-897c-e6a0439b0baf None admin] PATH_INFO: `/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/8da8d12ac74644f280ba31da1715f603/roles/5f8e9da552194885bbd7684f1a9c4f2d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:33.400081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:33.414341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63040444-4d4e-4779-897c-e6a0439b0baf None admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9e1d9331b4d4dbd3dd447937c42ad, user_id=8da8d12ac74644f280ba31da1715f603, role_id=5f8e9da552194885bbd7684f1a9c4f2d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:33.415757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63040444-4d4e-4779-897c-e6a0439b0baf None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:33.416147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f416e27-3ea0-4ab0-b798-cf7ca2a82c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:33.416962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f416e27-3ea0-4ab0-b798-cf7ca2a82c37 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:33.417372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f416e27-3ea0-4ab0-b798-cf7ca2a82c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:33.417766 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f416e27-3ea0-4ab0-b798-cf7ca2a82c37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:33.427193 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f416e27-3ea0-4ab0-b798-cf7ca2a82c37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:33.429568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f416e27-3ea0-4ab0-b798-cf7ca2a82c37 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:33.432132 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 404/825] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:09:33 2022] PUT /identity/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/8da8d12ac74644f280ba31da1715f603/roles/5f8e9da552194885bbd7684f1a9c4f2d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:09:33.432998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63040444-4d4e-4779-897c-e6a0439b0baf None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-2f416e27-3ea0-4ab0-b798-cf7ca2a82c37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:33.467642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:33.469232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:33.470804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:33.470804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:33.470804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:33.470804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:33.470804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:33.470804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:33.470804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:33.470804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:33.475335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 422/826] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:33.477244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f416e27-3ea0-4ab0-b798-cf7ca2a82c37 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:33.518027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:33.574919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63040444-4d4e-4779-897c-e6a0439b0baf None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:33.615312 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6be375-ecc0-45cc-b66b-b2d66bc4f269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:33.616168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6be375-ecc0-45cc-b66b-b2d66bc4f269 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:33.616547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6be375-ecc0-45cc-b66b-b2d66bc4f269 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:33.616922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6be375-ecc0-45cc-b66b-b2d66bc4f269 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:33.617722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6be375-ecc0-45cc-b66b-b2d66bc4f269 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:33.618094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6be375-ecc0-45cc-b66b-b2d66bc4f269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:33.619605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6be375-ecc0-45cc-b66b-b2d66bc4f269 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:33.624824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f416e27-3ea0-4ab0-b798-cf7ca2a82c37 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:33.629566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 405/827] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Sep 21 16:09:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:33.629566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6be375-ecc0-45cc-b66b-b2d66bc4f269 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:33.679097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a74384-9403-44f8-b3d2-3d979fce5064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:33.680650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a74384-9403-44f8-b3d2-3d979fce5064 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:33.681309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a74384-9403-44f8-b3d2-3d979fce5064 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:33.682130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a74384-9403-44f8-b3d2-3d979fce5064 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:33.683697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a74384-9403-44f8-b3d2-3d979fce5064 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:33.686514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a74384-9403-44f8-b3d2-3d979fce5064 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:33.737397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 423/828] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:09:33 2022] POST /identity/v3/endpoints => generated 337 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:09:33.740414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a74384-9403-44f8-b3d2-3d979fce5064 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:33.798231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6be375-ecc0-45cc-b66b-b2d66bc4f269 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:33.823431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:33.865694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:33.912097 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d702a-3f49-406b-89a8-a086c750da14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:33.913468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d702a-3f49-406b-89a8-a086c750da14 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:33.913873 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d702a-3f49-406b-89a8-a086c750da14 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:33.914317 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d702a-3f49-406b-89a8-a086c750da14 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:33.915190 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d702a-3f49-406b-89a8-a086c750da14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:33.915632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d702a-3f49-406b-89a8-a086c750da14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:33.917288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d702a-3f49-406b-89a8-a086c750da14 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:33.941220 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 406/829] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:33.952146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d702a-3f49-406b-89a8-a086c750da14 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:33.971052 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a74384-9403-44f8-b3d2-3d979fce5064 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:34.042428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd074dd-2bc1-4996-a6ec-a06601965b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:34.042748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd074dd-2bc1-4996-a6ec-a06601965b2a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:34.043107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd074dd-2bc1-4996-a6ec-a06601965b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:34.043560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd074dd-2bc1-4996-a6ec-a06601965b2a None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:34.044637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd074dd-2bc1-4996-a6ec-a06601965b2a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:34.045142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd074dd-2bc1-4996-a6ec-a06601965b2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:34.047847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd074dd-2bc1-4996-a6ec-a06601965b2a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:34.059263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 424/830] 10.222.0.64 () {56 vars in 1124 bytes} [Wed Sep 21 16:09:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:34.067157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd074dd-2bc1-4996-a6ec-a06601965b2a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:34.184578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d702a-3f49-406b-89a8-a086c750da14 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:34.259282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a61d5b-8afc-49d3-94bc-53cc5a8dd548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:34.260369 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a61d5b-8afc-49d3-94bc-53cc5a8dd548 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:34.260831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a61d5b-8afc-49d3-94bc-53cc5a8dd548 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:34.261298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a61d5b-8afc-49d3-94bc-53cc5a8dd548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:34.262718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a61d5b-8afc-49d3-94bc-53cc5a8dd548 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=f0acf23d224e4b16ad03cdd9bf7cae8e) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:34.263238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a61d5b-8afc-49d3-94bc-53cc5a8dd548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:34.265788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a61d5b-8afc-49d3-94bc-53cc5a8dd548 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:34.288848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 407/831] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:09:33 2022] GET /identity/v3/role_assignments?user.id=f0acf23d224e4b16ad03cdd9bf7cae8e&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=459d01a218314543a7a8329e88b78331 => generated 608 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:34.297259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd074dd-2bc1-4996-a6ec-a06601965b2a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:34.301035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:34.376620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e303de4-0fb4-4e2e-9cc6-89ed0e872d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:34.377493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e303de4-0fb4-4e2e-9cc6-89ed0e872d00 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:34.377956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e303de4-0fb4-4e2e-9cc6-89ed0e872d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:34.378449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e303de4-0fb4-4e2e-9cc6-89ed0e872d00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:34.379713 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e303de4-0fb4-4e2e-9cc6-89ed0e872d00 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:34.382604 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e303de4-0fb4-4e2e-9cc6-89ed0e872d00 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:34.419574 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 425/832] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:09:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:09:36.244010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c81621-0e5f-4026-a050-553657df9578 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:36.244879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c81621-0e5f-4026-a050-553657df9578 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:36.245011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c81621-0e5f-4026-a050-553657df9578 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:36.247921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 408/833] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:36.257813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879fd24-71bb-4785-8df3-866a087ba65f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:36.258667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879fd24-71bb-4785-8df3-866a087ba65f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:36.258804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879fd24-71bb-4785-8df3-866a087ba65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:36.289952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4879fd24-71bb-4785-8df3-866a087ba65f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:36.291594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4879fd24-71bb-4785-8df3-866a087ba65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:36.427558 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 426/834] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:36.774251 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c81621-0e5f-4026-a050-553657df9578 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:36.840119 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:36.939501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c81621-0e5f-4026-a050-553657df9578 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:37.011603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651b211-eeb8-413f-b258-b73ea2ccdc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:37.012680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651b211-eeb8-413f-b258-b73ea2ccdc15 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:37.013165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651b211-eeb8-413f-b258-b73ea2ccdc15 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:37.014009 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651b211-eeb8-413f-b258-b73ea2ccdc15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:37.036193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651b211-eeb8-413f-b258-b73ea2ccdc15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-6651b211-eeb8-413f-b258-b73ea2ccdc15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:37.045580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:37.046750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:37.047652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:37.047652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:37.047652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:37.047652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:37.047652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:37.047652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:37.047652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:37.047652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:37.047652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:37.047652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:37.047652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:09:37.047652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:37.048339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 409/835] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:09:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:37.057428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879fd24-71bb-4785-8df3-866a087ba65f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:37.170747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:37.189162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879fd24-71bb-4785-8df3-866a087ba65f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:37.231352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1426c3b-a4da-4fbf-88d6-bf5b5ef300e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:37.231996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1426c3b-a4da-4fbf-88d6-bf5b5ef300e9 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:37.232277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1426c3b-a4da-4fbf-88d6-bf5b5ef300e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:37.232553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1426c3b-a4da-4fbf-88d6-bf5b5ef300e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:37.233282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1426c3b-a4da-4fbf-88d6-bf5b5ef300e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:37.233592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1426c3b-a4da-4fbf-88d6-bf5b5ef300e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:37.235066 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1426c3b-a4da-4fbf-88d6-bf5b5ef300e9 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:37.246141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 427/836] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:09:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:37.255527 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651b211-eeb8-413f-b258-b73ea2ccdc15 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:37.262183 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dfcf9-7b73-47da-b512-ea47d9aa22e8 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:37.262702 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dfcf9-7b73-47da-b512-ea47d9aa22e8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:37.262794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548dfcf9-7b73-47da-b512-ea47d9aa22e8 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:37.264397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 428/837] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:09:37.276852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca49758-65de-4446-bb3d-814988071e07 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:37.277989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca49758-65de-4446-bb3d-814988071e07 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:37.278763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca49758-65de-4446-bb3d-814988071e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:37.309126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6ca49758-65de-4446-bb3d-814988071e07 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:37.310464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6ca49758-65de-4446-bb3d-814988071e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:37.407155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 429/838] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:09:37.408660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651b211-eeb8-413f-b258-b73ea2ccdc15 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:37.423276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:37.454598 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f145bd-4687-4f3e-a0c7-e58a48738eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:37.455294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f145bd-4687-4f3e-a0c7-e58a48738eab None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:37.455581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f145bd-4687-4f3e-a0c7-e58a48738eab None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:37.455855 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f145bd-4687-4f3e-a0c7-e58a48738eab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:37.470949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f145bd-4687-4f3e-a0c7-e58a48738eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:37.472616 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f145bd-4687-4f3e-a0c7-e58a48738eab None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b1f145bd-4687-4f3e-a0c7-e58a48738eab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:37.490936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:37.491908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:37.493230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:37.493230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:37.493230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:37.493230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 410/839] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:37.498319 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca49758-65de-4446-bb3d-814988071e07 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:37.581731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775cec0-f9c1-44ec-a081-02955b638905 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:37.581917 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775cec0-f9c1-44ec-a081-02955b638905 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:37.582229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775cec0-f9c1-44ec-a081-02955b638905 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:37.583843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 411/840] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:09:37.590237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026c34d-3b7b-495e-9a71-00d2b9b1ebca None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:37.590848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026c34d-3b7b-495e-9a71-00d2b9b1ebca None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:37.591349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026c34d-3b7b-495e-9a71-00d2b9b1ebca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:37.593559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:37.615013 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a026c34d-3b7b-495e-9a71-00d2b9b1ebca None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:37.616468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a026c34d-3b7b-495e-9a71-00d2b9b1ebca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:37.624489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca49758-65de-4446-bb3d-814988071e07 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:37.673781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cfc17-9287-48cd-843e-23b7c65b2281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:37.674732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cfc17-9287-48cd-843e-23b7c65b2281 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:37.675216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cfc17-9287-48cd-843e-23b7c65b2281 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:37.675677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718cfc17-9287-48cd-843e-23b7c65b2281 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:37.676524 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cfc17-9287-48cd-843e-23b7c65b2281 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:37.677014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cfc17-9287-48cd-843e-23b7c65b2281 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:37.679961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718cfc17-9287-48cd-843e-23b7c65b2281 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:37.704618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 430/841] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:37.707380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cfc17-9287-48cd-843e-23b7c65b2281 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:37.729904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 412/842] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:37.839062 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718cfc17-9287-48cd-843e-23b7c65b2281 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:37.846118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efacbf-604e-4616-9645-6e4a0eaa45b5 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:37.846420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efacbf-604e-4616-9645-6e4a0eaa45b5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:37.846681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52efacbf-604e-4616-9645-6e4a0eaa45b5 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:37.849485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 413/843] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:37.855131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca152fd-b28d-4041-92af-7507f7961439 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:37.855795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca152fd-b28d-4041-92af-7507f7961439 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:37.856300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca152fd-b28d-4041-92af-7507f7961439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:37.879185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fca152fd-b28d-4041-92af-7507f7961439 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:37.880430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fca152fd-b28d-4041-92af-7507f7961439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:37.882734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4756f-e419-40ca-91c1-213798347ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:37.883741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4756f-e419-40ca-91c1-213798347ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:37.884361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4756f-e419-40ca-91c1-213798347ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:37.884635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4756f-e419-40ca-91c1-213798347ea4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:37.889741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:37.900371 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4756f-e419-40ca-91c1-213798347ea4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:37.901840 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4756f-e419-40ca-91c1-213798347ea4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6fc4756f-e419-40ca-91c1-213798347ea4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:37.914978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:37.916042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:37.916985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:37.916985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:37.916985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:37.916985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:37.916985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:37.916985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:37.916985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:37.916985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:37.916985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 21 16:09:37.916985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:37.921406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 431/844] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:09:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:37.922426 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4756f-e419-40ca-91c1-213798347ea4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:37.986077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 414/845] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:37.986077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca152fd-b28d-4041-92af-7507f7961439 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:38.008685 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:38.109699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4756f-e419-40ca-91c1-213798347ea4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:38.115543 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca152fd-b28d-4041-92af-7507f7961439 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:38.158689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36a8ec-d61c-4e71-9486-282a1f79e0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:38.159343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36a8ec-d61c-4e71-9486-282a1f79e0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:38.159616 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36a8ec-d61c-4e71-9486-282a1f79e0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:38.159883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36a8ec-d61c-4e71-9486-282a1f79e0f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:38.160103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fa4b8-ec3e-4d32-a420-7a682cf6090a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:38.161106 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa4b8-ec3e-4d32-a420-7a682cf6090a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:38.161597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa4b8-ec3e-4d32-a420-7a682cf6090a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:38.162121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa4b8-ec3e-4d32-a420-7a682cf6090a None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:38.164079 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804fa4b8-ec3e-4d32-a420-7a682cf6090a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:38.164635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804fa4b8-ec3e-4d32-a420-7a682cf6090a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:38.167435 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804fa4b8-ec3e-4d32-a420-7a682cf6090a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:38.174091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36a8ec-d61c-4e71-9486-282a1f79e0f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ed36a8ec-d61c-4e71-9486-282a1f79e0f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:38.180021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:38.181112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:38.181942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:38.181942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:38.181942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:38.181942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:38.181942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:38.181942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:38.181942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:38.181942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:38.181942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:38.181942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:38.181942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:09:38.181942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:38.186385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 415/846] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:09:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:38.187893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36a8ec-d61c-4e71-9486-282a1f79e0f2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:38.202345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 432/847] 10.222.0.64 () {56 vars in 1135 bytes} [Wed Sep 21 16:09:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:38.202345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fa4b8-ec3e-4d32-a420-7a682cf6090a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:38.263855 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:38.316835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fa4b8-ec3e-4d32-a420-7a682cf6090a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:38.329216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:38.353293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36a8ec-d61c-4e71-9486-282a1f79e0f2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:38.366607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70d2c2-312b-439f-87cb-1b68f0fdef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:38.367135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70d2c2-312b-439f-87cb-1b68f0fdef50 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:38.367412 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70d2c2-312b-439f-87cb-1b68f0fdef50 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:38.367682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70d2c2-312b-439f-87cb-1b68f0fdef50 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:38.368348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70d2c2-312b-439f-87cb-1b68f0fdef50 None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:38.369021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70d2c2-312b-439f-87cb-1b68f0fdef50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:38.370018 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70d2c2-312b-439f-87cb-1b68f0fdef50 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:38.378520 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 433/848] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:09:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:38.379377 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70d2c2-312b-439f-87cb-1b68f0fdef50 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:38.396513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147ecb6-9aa6-46b2-a02f-29c3b2976676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:38.397443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147ecb6-9aa6-46b2-a02f-29c3b2976676 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:38.398300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147ecb6-9aa6-46b2-a02f-29c3b2976676 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:38.398644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b147ecb6-9aa6-46b2-a02f-29c3b2976676 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:38.399538 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147ecb6-9aa6-46b2-a02f-29c3b2976676 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:38.401089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b147ecb6-9aa6-46b2-a02f-29c3b2976676 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b147ecb6-9aa6-46b2-a02f-29c3b2976676 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:38.411988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:38.414004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:38.416524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 416/849] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:09:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:38.418376 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147ecb6-9aa6-46b2-a02f-29c3b2976676 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:38.515099 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70d2c2-312b-439f-87cb-1b68f0fdef50 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:38.532416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:38.544485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b147ecb6-9aa6-46b2-a02f-29c3b2976676 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:38.551539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfbfcc-2ce8-4624-ac9d-0d3c067f7f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:38.552164 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfbfcc-2ce8-4624-ac9d-0d3c067f7f95 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:38.552441 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfbfcc-2ce8-4624-ac9d-0d3c067f7f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:38.552709 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bfbfcc-2ce8-4624-ac9d-0d3c067f7f95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:38.585408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc3854-afd8-4eb6-b259-2b0cabc2e9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:38.586041 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc3854-afd8-4eb6-b259-2b0cabc2e9ba None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:38.586352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc3854-afd8-4eb6-b259-2b0cabc2e9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:38.586818 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc3854-afd8-4eb6-b259-2b0cabc2e9ba None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:38.587422 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc3854-afd8-4eb6-b259-2b0cabc2e9ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:38.588836 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc3854-afd8-4eb6-b259-2b0cabc2e9ba None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-4cfc3854-afd8-4eb6-b259-2b0cabc2e9ba None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:38.598974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:38.601215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:38.602389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 417/850] 10.222.0.64 () {56 vars in 1132 bytes} [Wed Sep 21 16:09:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:38.603300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc3854-afd8-4eb6-b259-2b0cabc2e9ba None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:38.619962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:38.645927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfbfcc-2ce8-4624-ac9d-0d3c067f7f95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:38.647509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bfbfcc-2ce8-4624-ac9d-0d3c067f7f95 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:38.738272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc3854-afd8-4eb6-b259-2b0cabc2e9ba None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:38.774443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 434/851] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:09:38 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:09:38.775685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfbfcc-2ce8-4624-ac9d-0d3c067f7f95 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:38.784742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ace9c1-9f1c-4113-b711-2ad9338756fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:38.785406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ace9c1-9f1c-4113-b711-2ad9338756fb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:38.785696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ace9c1-9f1c-4113-b711-2ad9338756fb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:38.785944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ace9c1-9f1c-4113-b711-2ad9338756fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:38.799123 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ace9c1-9f1c-4113-b711-2ad9338756fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:38.800710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ace9c1-9f1c-4113-b711-2ad9338756fb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:38.802309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-83ace9c1-9f1c-4113-b711-2ad9338756fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:38.819469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:38.821512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:38.823907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:38.823907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:38.823907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:38.823907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 418/852] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:38.823907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ace9c1-9f1c-4113-b711-2ad9338756fb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:38.893957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:38.915425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bfbfcc-2ce8-4624-ac9d-0d3c067f7f95 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:38.945692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ace9c1-9f1c-4113-b711-2ad9338756fb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:38.975636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add5eb5-4f94-44dc-a701-b6b11d4926d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:38.976647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add5eb5-4f94-44dc-a701-b6b11d4926d2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:38.977151 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add5eb5-4f94-44dc-a701-b6b11d4926d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:38.977627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add5eb5-4f94-44dc-a701-b6b11d4926d2 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:38.978818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add5eb5-4f94-44dc-a701-b6b11d4926d2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:38.979415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add5eb5-4f94-44dc-a701-b6b11d4926d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:38.981895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add5eb5-4f94-44dc-a701-b6b11d4926d2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:38.986239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec07afb-0439-454f-bb70-34f19cf6120a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:38.987221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec07afb-0439-454f-bb70-34f19cf6120a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:38.987672 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec07afb-0439-454f-bb70-34f19cf6120a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:38.988130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec07afb-0439-454f-bb70-34f19cf6120a None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:38.989155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec07afb-0439-454f-bb70-34f19cf6120a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:38.989639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec07afb-0439-454f-bb70-34f19cf6120a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:38.997722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 435/853] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:09:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:38.998010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec07afb-0439-454f-bb70-34f19cf6120a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:38.998200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add5eb5-4f94-44dc-a701-b6b11d4926d2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:39.000472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 419/854] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Sep 21 16:09:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:39.001417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec07afb-0439-454f-bb70-34f19cf6120a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:39.101938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:39.126323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add5eb5-4f94-44dc-a701-b6b11d4926d2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:39.148709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec07afb-0439-454f-bb70-34f19cf6120a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:39.168300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752b34b-9d86-445e-b640-45a791cbd479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:39.168946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752b34b-9d86-445e-b640-45a791cbd479 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:39.169223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752b34b-9d86-445e-b640-45a791cbd479 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:39.169500 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752b34b-9d86-445e-b640-45a791cbd479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:39.179286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:39.181214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752b34b-9d86-445e-b640-45a791cbd479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:39.182719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752b34b-9d86-445e-b640-45a791cbd479 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:39.193089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ea82f-7a55-4fa3-b03a-794f9458d8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:39.194290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ea82f-7a55-4fa3-b03a-794f9458d8ee None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:39.194582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ea82f-7a55-4fa3-b03a-794f9458d8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:39.194841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ea82f-7a55-4fa3-b03a-794f9458d8ee None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:39.195488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ea82f-7a55-4fa3-b03a-794f9458d8ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:39.195785 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ea82f-7a55-4fa3-b03a-794f9458d8ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:39.197140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ea82f-7a55-4fa3-b03a-794f9458d8ee None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6752b34b-9d86-445e-b640-45a791cbd479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:39.205162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:39.207299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:39.209073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:39.209073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:39.209073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:39.209073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 436/855] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:39.209073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752b34b-9d86-445e-b640-45a791cbd479 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:39.217843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 420/856] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:39.219317 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ea82f-7a55-4fa3-b03a-794f9458d8ee None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:39.333013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752b34b-9d86-445e-b640-45a791cbd479 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:39.334290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ea82f-7a55-4fa3-b03a-794f9458d8ee None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:39.374251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:39.377984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef084634-b8ae-4559-9a60-4d6460d5851b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:39.378627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef084634-b8ae-4559-9a60-4d6460d5851b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:39.378966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef084634-b8ae-4559-9a60-4d6460d5851b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:39.379264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef084634-b8ae-4559-9a60-4d6460d5851b None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:39.380092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef084634-b8ae-4559-9a60-4d6460d5851b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:39.380513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef084634-b8ae-4559-9a60-4d6460d5851b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:39.380675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47efa18-29ce-4f52-bf3e-8be5e8ae6f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:39.381287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47efa18-29ce-4f52-bf3e-8be5e8ae6f66 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:39.381566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47efa18-29ce-4f52-bf3e-8be5e8ae6f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:39.381837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47efa18-29ce-4f52-bf3e-8be5e8ae6f66 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:39.382483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef084634-b8ae-4559-9a60-4d6460d5851b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:39.382650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47efa18-29ce-4f52-bf3e-8be5e8ae6f66 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:39.382817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47efa18-29ce-4f52-bf3e-8be5e8ae6f66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:39.384153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47efa18-29ce-4f52-bf3e-8be5e8ae6f66 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:39.388926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 437/857] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:09:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:39.389750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47efa18-29ce-4f52-bf3e-8be5e8ae6f66 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:39.402199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 421/858] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Sep 21 16:09:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:39.402199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef084634-b8ae-4559-9a60-4d6460d5851b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:39.511459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:39.615111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47efa18-29ce-4f52-bf3e-8be5e8ae6f66 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:39.622816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef084634-b8ae-4559-9a60-4d6460d5851b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:39.686176 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cc234-7deb-4623-a524-d9344d1a803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:39.686176 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cc234-7deb-4623-a524-d9344d1a803b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:39.686176 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cc234-7deb-4623-a524-d9344d1a803b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:39.686706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09cc234-7deb-4623-a524-d9344d1a803b None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:39.688111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cc234-7deb-4623-a524-d9344d1a803b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:39.688111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cc234-7deb-4623-a524-d9344d1a803b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:39.691409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09cc234-7deb-4623-a524-d9344d1a803b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:39.704628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4873224-cac4-4f16-a7f4-4f0d30a3e7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:39.705642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4873224-cac4-4f16-a7f4-4f0d30a3e7ad None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:39.706167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4873224-cac4-4f16-a7f4-4f0d30a3e7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:39.706613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4873224-cac4-4f16-a7f4-4f0d30a3e7ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:39.714328 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 438/859] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:39.725884 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cc234-7deb-4623-a524-d9344d1a803b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:39.726404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4873224-cac4-4f16-a7f4-4f0d30a3e7ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:39.729092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4873224-cac4-4f16-a7f4-4f0d30a3e7ad None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b4873224-cac4-4f16-a7f4-4f0d30a3e7ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:39.755027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:39.756656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:39.757590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:39.757590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:39.757590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:39.757590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:39.757590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:39.757590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:39.757590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:39.757590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:39.757590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 16:09:39.757590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:39.758177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 422/860] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Sep 21 16:09:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:39.768495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4873224-cac4-4f16-a7f4-4f0d30a3e7ad None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:39.835826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:39.937315 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09cc234-7deb-4623-a524-d9344d1a803b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:39.991405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4873224-cac4-4f16-a7f4-4f0d30a3e7ad None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:40.015348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:40.020930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5340d1-914b-40a4-8655-9a3b739524b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:40.023440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5340d1-914b-40a4-8655-9a3b739524b8 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:40.024080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5340d1-914b-40a4-8655-9a3b739524b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:40.024636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5340d1-914b-40a4-8655-9a3b739524b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:40.040984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5340d1-914b-40a4-8655-9a3b739524b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:40.042473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5340d1-914b-40a4-8655-9a3b739524b8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ef5340d1-914b-40a4-8655-9a3b739524b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:40.055037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:40.056531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:40.058578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:40.058578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:40.058578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:40.058578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:40.058578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:40.058578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:40.058578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:40.058578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:40.061302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 439/861] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:40.062819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5340d1-914b-40a4-8655-9a3b739524b8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:40.085257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192b916-63aa-42ed-bfa3-bc03d007035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:40.087332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192b916-63aa-42ed-bfa3-bc03d007035f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:40.088058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192b916-63aa-42ed-bfa3-bc03d007035f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:40.088724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192b916-63aa-42ed-bfa3-bc03d007035f None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:40.091592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192b916-63aa-42ed-bfa3-bc03d007035f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:40.092745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192b916-63aa-42ed-bfa3-bc03d007035f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:40.097782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192b916-63aa-42ed-bfa3-bc03d007035f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:40.152884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 423/862] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Sep 21 16:09:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:40.154064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192b916-63aa-42ed-bfa3-bc03d007035f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:40.194572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5340d1-914b-40a4-8655-9a3b739524b8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:40.216975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:40.241057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4ae7d-f958-4704-b0d1-19c862e7eb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:40.242666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4ae7d-f958-4704-b0d1-19c862e7eb07 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:40.243126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4ae7d-f958-4704-b0d1-19c862e7eb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:40.243566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4ae7d-f958-4704-b0d1-19c862e7eb07 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:40.244439 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4ae7d-f958-4704-b0d1-19c862e7eb07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:40.244900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4ae7d-f958-4704-b0d1-19c862e7eb07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:40.246571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4ae7d-f958-4704-b0d1-19c862e7eb07 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:40.264255 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 440/863] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:40.270354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4ae7d-f958-4704-b0d1-19c862e7eb07 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:40.325234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192b916-63aa-42ed-bfa3-bc03d007035f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:40.390307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ddd2c-d4ca-4cc2-8319-1c9ceb7a0f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:40.392937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ddd2c-d4ca-4cc2-8319-1c9ceb7a0f0f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:40.393634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ddd2c-d4ca-4cc2-8319-1c9ceb7a0f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:40.394468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199ddd2c-d4ca-4cc2-8319-1c9ceb7a0f0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:40.403131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4ae7d-f958-4704-b0d1-19c862e7eb07 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:40.449009 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:40.456149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85bb38-f71b-4926-9971-704a915476c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:40.457336 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85bb38-f71b-4926-9971-704a915476c6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:40.457961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85bb38-f71b-4926-9971-704a915476c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:40.458661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85bb38-f71b-4926-9971-704a915476c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:40.460906 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85bb38-f71b-4926-9971-704a915476c6 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=d71a1b8081ce490db46db95f14024ba6) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:40.461594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85bb38-f71b-4926-9971-704a915476c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:40.464322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85bb38-f71b-4926-9971-704a915476c6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:40.482080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 441/864] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:09:40 2022] GET /identity/v3/role_assignments?user.id=d71a1b8081ce490db46db95f14024ba6&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=459d01a218314543a7a8329e88b78331 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:40.525219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ddd2c-d4ca-4cc2-8319-1c9ceb7a0f0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:40.526850 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199ddd2c-d4ca-4cc2-8319-1c9ceb7a0f0f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:40.681659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 424/865] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:09:40 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:09:40.683462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85bb38-f71b-4926-9971-704a915476c6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:40.734619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:40.820186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85bb38-f71b-4926-9971-704a915476c6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:40.875474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9e18f-4b5e-4633-adcf-a91d744bdf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:40.876189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9e18f-4b5e-4633-adcf-a91d744bdf0c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:40.876620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9e18f-4b5e-4633-adcf-a91d744bdf0c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:40.876928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9e18f-4b5e-4633-adcf-a91d744bdf0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:40.893382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9e18f-4b5e-4633-adcf-a91d744bdf0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:40.894950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9e18f-4b5e-4633-adcf-a91d744bdf0c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-34f9e18f-4b5e-4633-adcf-a91d744bdf0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:40.921770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:40.922851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:40.923804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:40.923804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:40.923804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:40.929688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 442/866] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:40.932638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ddd2c-d4ca-4cc2-8319-1c9ceb7a0f0f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:41.049129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:41.110545 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199ddd2c-d4ca-4cc2-8319-1c9ceb7a0f0f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:41.170565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028aab27-87a1-4f76-a64a-b781753153ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:41.171559 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028aab27-87a1-4f76-a64a-b781753153ef None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:41.172073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028aab27-87a1-4f76-a64a-b781753153ef None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:41.172570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028aab27-87a1-4f76-a64a-b781753153ef None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:41.173523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028aab27-87a1-4f76-a64a-b781753153ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:41.174078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028aab27-87a1-4f76-a64a-b781753153ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:41.176000 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028aab27-87a1-4f76-a64a-b781753153ef None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:41.202215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 425/867] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:41.204352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9e18f-4b5e-4633-adcf-a91d744bdf0c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:41.319744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:41.337438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9e18f-4b5e-4633-adcf-a91d744bdf0c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:41.402615 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed95c25-8553-4542-943a-e36ecc2ba60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:41.403729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed95c25-8553-4542-943a-e36ecc2ba60f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:41.404361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed95c25-8553-4542-943a-e36ecc2ba60f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:41.404803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed95c25-8553-4542-943a-e36ecc2ba60f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:41.417816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed95c25-8553-4542-943a-e36ecc2ba60f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:41.420495 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed95c25-8553-4542-943a-e36ecc2ba60f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-1ed95c25-8553-4542-943a-e36ecc2ba60f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:41.443078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:41.444081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:41.444961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:41.444961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:41.444961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:41.444961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:41.444961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:41.444961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:41.444961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:41.444961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:41.450192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 443/868] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:41.455841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028aab27-87a1-4f76-a64a-b781753153ef None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:41.657931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028aab27-87a1-4f76-a64a-b781753153ef None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:41.708111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:41.725159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c006119-b11c-4cb7-9550-a8802d8c7b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:41.726334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c006119-b11c-4cb7-9550-a8802d8c7b54 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:41.726837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c006119-b11c-4cb7-9550-a8802d8c7b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:41.727557 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c006119-b11c-4cb7-9550-a8802d8c7b54 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:41.731148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c006119-b11c-4cb7-9550-a8802d8c7b54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:41.731711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c006119-b11c-4cb7-9550-a8802d8c7b54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:41.734563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c006119-b11c-4cb7-9550-a8802d8c7b54 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:41.766432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 426/869] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:41.773181 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed95c25-8553-4542-943a-e36ecc2ba60f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:41.978661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9b70-77ad-4928-b0fd-73ebc1939e1e None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:41.978975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9b70-77ad-4928-b0fd-73ebc1939e1e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:41.979189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9b70-77ad-4928-b0fd-73ebc1939e1e None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:41.980912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 427/870] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:09:41.983360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed95c25-8553-4542-943a-e36ecc2ba60f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:41.987379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ea222-e28e-45e1-89e6-1119f1ba4e34 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:41.987379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ea222-e28e-45e1-89e6-1119f1ba4e34 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:41.987379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ea222-e28e-45e1-89e6-1119f1ba4e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:42.009081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-439ea222-e28e-45e1-89e6-1119f1ba4e34 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:42.009859 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-439ea222-e28e-45e1-89e6-1119f1ba4e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:42.025719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fd11d-1895-4725-9e22-e79e65ab2532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:42.026747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd11d-1895-4725-9e22-e79e65ab2532 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:42.027269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd11d-1895-4725-9e22-e79e65ab2532 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:42.027824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fd11d-1895-4725-9e22-e79e65ab2532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:42.029074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fd11d-1895-4725-9e22-e79e65ab2532 None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=8da8d12ac74644f280ba31da1715f603) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:42.029617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fd11d-1895-4725-9e22-e79e65ab2532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:42.031420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fd11d-1895-4725-9e22-e79e65ab2532 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:42.039301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 444/871] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:09:41 2022] GET /identity/v3/role_assignments?user.id=8da8d12ac74644f280ba31da1715f603&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:42.093325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 428/872] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:42.097217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263016-09cc-49cb-b129-053f7d907943 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:42.097481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263016-09cc-49cb-b129-053f7d907943 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:42.097820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1263016-09cc-49cb-b129-053f7d907943 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:42.099547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 445/873] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:42.108497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d7980-bd87-4816-a1e3-2ae2466363ee None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:42.108800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d7980-bd87-4816-a1e3-2ae2466363ee None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:42.109085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84d7980-bd87-4816-a1e3-2ae2466363ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:42.129724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c84d7980-bd87-4816-a1e3-2ae2466363ee None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:42.131081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c84d7980-bd87-4816-a1e3-2ae2466363ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:42.211112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 429/874] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:42.418532 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263016-09cc-49cb-b129-053f7d907943 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:42.439232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:42.556338 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1263016-09cc-49cb-b129-053f7d907943 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:42.623812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32f1e45-5b09-42ad-9409-20376d233215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:42.625327 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f1e45-5b09-42ad-9409-20376d233215 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:42.626158 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f1e45-5b09-42ad-9409-20376d233215 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:42.626989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32f1e45-5b09-42ad-9409-20376d233215 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:42.628483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f1e45-5b09-42ad-9409-20376d233215 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:42.631907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32f1e45-5b09-42ad-9409-20376d233215 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:42.635385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d7980-bd87-4816-a1e3-2ae2466363ee None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:42.664029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 446/875] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:09:42 2022] POST /identity/v3/services => generated 269 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:09:42.703033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:42.774797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84d7980-bd87-4816-a1e3-2ae2466363ee None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:42.824466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ee03d-ae9e-4404-9f2c-7064abe798f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:42.825146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ee03d-ae9e-4404-9f2c-7064abe798f0 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:42.825431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ee03d-ae9e-4404-9f2c-7064abe798f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:42.825710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ee03d-ae9e-4404-9f2c-7064abe798f0 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:42.826439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ee03d-ae9e-4404-9f2c-7064abe798f0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:42.827895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ee03d-ae9e-4404-9f2c-7064abe798f0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:42.853751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 430/876] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:09:42 2022] POST /identity/v3/services => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:09:43.201262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804e648-1da4-4ebd-ab1f-f64b20ab3a11 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:43.201951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804e648-1da4-4ebd-ab1f-f64b20ab3a11 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:43.202460 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804e648-1da4-4ebd-ab1f-f64b20ab3a11 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:43.204274 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 447/877] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:43.210280 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a064ac6-e008-4569-bb92-9efac1646db2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:43.210876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a064ac6-e008-4569-bb92-9efac1646db2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:43.211367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a064ac6-e008-4569-bb92-9efac1646db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:43.231054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0a064ac6-e008-4569-bb92-9efac1646db2 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:43.232587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0a064ac6-e008-4569-bb92-9efac1646db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:43.319379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 431/878] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:43.600059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804e648-1da4-4ebd-ab1f-f64b20ab3a11 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:43.691838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:43.721231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804e648-1da4-4ebd-ab1f-f64b20ab3a11 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:43.766259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff00a27-cbe5-4a7e-8790-f05cdf975e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:43.766843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff00a27-cbe5-4a7e-8790-f05cdf975e26 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:43.767120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff00a27-cbe5-4a7e-8790-f05cdf975e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:43.767392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff00a27-cbe5-4a7e-8790-f05cdf975e26 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:43.767992 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff00a27-cbe5-4a7e-8790-f05cdf975e26 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:43.769388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff00a27-cbe5-4a7e-8790-f05cdf975e26 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-bff00a27-cbe5-4a7e-8790-f05cdf975e26 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:43.780800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:43.781860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:43.782740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 448/879] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Sep 21 16:09:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:43.790836 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a064ac6-e008-4569-bb92-9efac1646db2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:43.828426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:43.915237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a064ac6-e008-4569-bb92-9efac1646db2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:43.956199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9331b5a6-76f9-49c5-b0dd-e447f6c485ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:43.957178 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331b5a6-76f9-49c5-b0dd-e447f6c485ce None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:43.957655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331b5a6-76f9-49c5-b0dd-e447f6c485ce None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:43.958193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9331b5a6-76f9-49c5-b0dd-e447f6c485ce None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:43.959187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9331b5a6-76f9-49c5-b0dd-e447f6c485ce None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:43.959703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9331b5a6-76f9-49c5-b0dd-e447f6c485ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:43.961286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9331b5a6-76f9-49c5-b0dd-e447f6c485ce None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:43.967384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 432/880] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Sep 21 16:09:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:43.974133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff00a27-cbe5-4a7e-8790-f05cdf975e26 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:44.110347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff00a27-cbe5-4a7e-8790-f05cdf975e26 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:44.150170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfaab9-c7e8-4407-904c-ff1aae6551e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:44.150781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfaab9-c7e8-4407-904c-ff1aae6551e6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:44.151060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfaab9-c7e8-4407-904c-ff1aae6551e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:44.151332 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfaab9-c7e8-4407-904c-ff1aae6551e6 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:44.151972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfaab9-c7e8-4407-904c-ff1aae6551e6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:44.152271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfaab9-c7e8-4407-904c-ff1aae6551e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:44.153623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfaab9-c7e8-4407-904c-ff1aae6551e6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:44.159381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 449/881] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Sep 21 16:09:43 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:44.512456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b5331-4dde-4a84-8e9e-28a0a8eb4ac1 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:44.513123 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b5331-4dde-4a84-8e9e-28a0a8eb4ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:44.513592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6b5331-4dde-4a84-8e9e-28a0a8eb4ac1 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:44.515547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 433/882] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:44.521026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5842b-72b9-453d-ad81-dd43271316a6 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:44.522216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5842b-72b9-453d-ad81-dd43271316a6 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:44.522759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5842b-72b9-453d-ad81-dd43271316a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:44.530663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:44.543010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ffc5842b-72b9-453d-ad81-dd43271316a6 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:44.544200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ffc5842b-72b9-453d-ad81-dd43271316a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:44.645136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 450/883] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:44.924910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b5331-4dde-4a84-8e9e-28a0a8eb4ac1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:44.969817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2db724-0592-4e51-9ee8-d59c642f2fd3 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:44.970395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2db724-0592-4e51-9ee8-d59c642f2fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:44.970862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2db724-0592-4e51-9ee8-d59c642f2fd3 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:44.972496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 451/884] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:44.978522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f74f5-7d23-4419-a072-fa29cb1892bf None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:44.979103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f74f5-7d23-4419-a072-fa29cb1892bf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:44.979559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f74f5-7d23-4419-a072-fa29cb1892bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:45.017842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f90f74f5-7d23-4419-a072-fa29cb1892bf None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:45.020707 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f90f74f5-7d23-4419-a072-fa29cb1892bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:45.053654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:45.065994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6b5331-4dde-4a84-8e9e-28a0a8eb4ac1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:45.116967 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581827a7-222a-4d8b-ab7f-b13097f47aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:45.118166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581827a7-222a-4d8b-ab7f-b13097f47aaf None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:45.118639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581827a7-222a-4d8b-ab7f-b13097f47aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:45.119223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581827a7-222a-4d8b-ab7f-b13097f47aaf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:45.141946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581827a7-222a-4d8b-ab7f-b13097f47aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:45.149219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 452/885] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-581827a7-222a-4d8b-ab7f-b13097f47aaf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:45.152242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:45.153208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:45.154076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:45.154076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:45.154076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:45.154076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:45.154076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:45.154076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:45.154076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:45.154076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:45.154076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:45.154076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:45.154076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 16:09:45.154076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:45.154893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 434/886] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Sep 21 16:09:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:45.160151 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f74f5-7d23-4419-a072-fa29cb1892bf None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:45.228879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:45.271663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f74f5-7d23-4419-a072-fa29cb1892bf None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:45.316785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8924d2e-d482-4842-be38-8e3075b8199b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:45.317417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8924d2e-d482-4842-be38-8e3075b8199b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:45.317694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8924d2e-d482-4842-be38-8e3075b8199b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:45.318016 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8924d2e-d482-4842-be38-8e3075b8199b None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:45.318752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8924d2e-d482-4842-be38-8e3075b8199b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:45.319056 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8924d2e-d482-4842-be38-8e3075b8199b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:45.320481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8924d2e-d482-4842-be38-8e3075b8199b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:45.330595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 453/887] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Sep 21 16:09:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:45.337920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581827a7-222a-4d8b-ab7f-b13097f47aaf None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:45.453759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8924d2e-d482-4842-be38-8e3075b8199b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:45.457359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581827a7-222a-4d8b-ab7f-b13097f47aaf None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:45.511092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ef004-24c8-4740-a0f9-8adf8fa269a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:45.511733 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ef004-24c8-4740-a0f9-8adf8fa269a3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:45.512120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ef004-24c8-4740-a0f9-8adf8fa269a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:45.512391 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ef004-24c8-4740-a0f9-8adf8fa269a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:45.526481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ef004-24c8-4740-a0f9-8adf8fa269a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:45.528080 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ef004-24c8-4740-a0f9-8adf8fa269a3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ca4ef004-24c8-4740-a0f9-8adf8fa269a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:45.546617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:45.548354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:45.550027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:45.550027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:45.550027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:45.550027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 435/888] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:09:45.550027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93397fd7-48d0-434e-baf2-346846bc7221 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:45.550027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93397fd7-48d0-434e-baf2-346846bc7221 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:45.551593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93397fd7-48d0-434e-baf2-346846bc7221 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:45.559222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 436/889] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:45.559222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93397fd7-48d0-434e-baf2-346846bc7221 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:45.561959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:45.614905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8924d2e-d482-4842-be38-8e3075b8199b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:45.669243 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:45.686983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7cbf93-9b76-47ab-a2bf-7d22a76e4402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:45.688075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cbf93-9b76-47ab-a2bf-7d22a76e4402 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:45.688578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cbf93-9b76-47ab-a2bf-7d22a76e4402 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:45.689085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7cbf93-9b76-47ab-a2bf-7d22a76e4402 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:45.690213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cbf93-9b76-47ab-a2bf-7d22a76e4402 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:45.692841 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7cbf93-9b76-47ab-a2bf-7d22a76e4402 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-1b7cbf93-9b76-47ab-a2bf-7d22a76e4402 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:45.712209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:45.713245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 16:09:45.714170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:45.714927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 454/890] 10.222.0.64 () {56 vars in 1132 bytes} [Wed Sep 21 16:09:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:45.716638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b843537-8ac2-45f2-9b06-c0ccf3a951c0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:45.717098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b843537-8ac2-45f2-9b06-c0ccf3a951c0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:45.717736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b843537-8ac2-45f2-9b06-c0ccf3a951c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:45.729934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93397fd7-48d0-434e-baf2-346846bc7221 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:45.757596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4b843537-8ac2-45f2-9b06-c0ccf3a951c0 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:45.759212 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4b843537-8ac2-45f2-9b06-c0ccf3a951c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:45.796215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe3ce6-906c-4d09-8582-51bf4d70c068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:45.797380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe3ce6-906c-4d09-8582-51bf4d70c068 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:45.797932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe3ce6-906c-4d09-8582-51bf4d70c068 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:45.798451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fe3ce6-906c-4d09-8582-51bf4d70c068 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:45.799545 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe3ce6-906c-4d09-8582-51bf4d70c068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:45.800056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe3ce6-906c-4d09-8582-51bf4d70c068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:45.802443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fe3ce6-906c-4d09-8582-51bf4d70c068 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:45.830278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 437/891] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:45.831824 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe3ce6-906c-4d09-8582-51bf4d70c068 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:45.862080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 455/892] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:45.862080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b843537-8ac2-45f2-9b06-c0ccf3a951c0 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:45.957276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:46.050716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fe3ce6-906c-4d09-8582-51bf4d70c068 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:46.075365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b843537-8ac2-45f2-9b06-c0ccf3a951c0 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:46.122342 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:46.126447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98633a08-a1db-4ead-a3a6-9af556888ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:46.127799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98633a08-a1db-4ead-a3a6-9af556888ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:46.128467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98633a08-a1db-4ead-a3a6-9af556888ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:46.129228 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98633a08-a1db-4ead-a3a6-9af556888ea0 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:46.130712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98633a08-a1db-4ead-a3a6-9af556888ea0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:46.131580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98633a08-a1db-4ead-a3a6-9af556888ea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:46.134566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98633a08-a1db-4ead-a3a6-9af556888ea0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:46.147700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 438/893] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Sep 21 16:09:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:46.154108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98633a08-a1db-4ead-a3a6-9af556888ea0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:46.161461 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdbccf-a1f0-41f5-9c45-de5e0f218200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:46.163108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdbccf-a1f0-41f5-9c45-de5e0f218200 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:46.163595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdbccf-a1f0-41f5-9c45-de5e0f218200 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:46.163938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbdbccf-a1f0-41f5-9c45-de5e0f218200 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:46.184730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdbccf-a1f0-41f5-9c45-de5e0f218200 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:46.187869 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbdbccf-a1f0-41f5-9c45-de5e0f218200 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6bbdbccf-a1f0-41f5-9c45-de5e0f218200 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:46.211305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:46.213147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:46.215019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:46.215019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:46.215019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:46.215019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:46.215019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:46.215019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:46.215019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:46.215019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:46.215019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 16:09:46.215019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:46.215019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 456/894] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Sep 21 16:09:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:46.215019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdbccf-a1f0-41f5-9c45-de5e0f218200 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:46.327695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98633a08-a1db-4ead-a3a6-9af556888ea0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:46.378366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:46.393380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbdbccf-a1f0-41f5-9c45-de5e0f218200 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:46.395252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d27ce-a0f6-4226-92da-2dd43e03eacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:46.396747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d27ce-a0f6-4226-92da-2dd43e03eacf None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:46.397607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d27ce-a0f6-4226-92da-2dd43e03eacf None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:46.398359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d27ce-a0f6-4226-92da-2dd43e03eacf None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:46.399508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d27ce-a0f6-4226-92da-2dd43e03eacf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:46.400039 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d27ce-a0f6-4226-92da-2dd43e03eacf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:46.402780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d27ce-a0f6-4226-92da-2dd43e03eacf None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:46.412413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 439/895] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Sep 21 16:09:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:46.413631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d27ce-a0f6-4226-92da-2dd43e03eacf None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:46.441770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea1022-d125-4de1-8df8-44d2f4091cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:46.442452 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea1022-d125-4de1-8df8-44d2f4091cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:46.442743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea1022-d125-4de1-8df8-44d2f4091cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:46.443543 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea1022-d125-4de1-8df8-44d2f4091cf2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:46.444415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea1022-d125-4de1-8df8-44d2f4091cf2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:46.445554 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea1022-d125-4de1-8df8-44d2f4091cf2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a8ea1022-d125-4de1-8df8-44d2f4091cf2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:46.460190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:46.462464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 16:09:46.464370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:46.468229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 457/896] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:09:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:46.469477 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea1022-d125-4de1-8df8-44d2f4091cf2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:46.545993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:46.598306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d27ce-a0f6-4226-92da-2dd43e03eacf None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:46.642214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8059b29-bed2-4758-988a-a27e9c1cd4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:46.643062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8059b29-bed2-4758-988a-a27e9c1cd4ff None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:46.643474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8059b29-bed2-4758-988a-a27e9c1cd4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:46.643889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8059b29-bed2-4758-988a-a27e9c1cd4ff None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:46.644056 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea1022-d125-4de1-8df8-44d2f4091cf2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:46.644895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8059b29-bed2-4758-988a-a27e9c1cd4ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:46.645346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8059b29-bed2-4758-988a-a27e9c1cd4ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:46.647876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8059b29-bed2-4758-988a-a27e9c1cd4ff None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:46.676826 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 440/897] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Sep 21 16:09:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:46.678347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8059b29-bed2-4758-988a-a27e9c1cd4ff None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:46.702578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd1e1b-1192-41e8-b0da-4d73035de127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:46.703598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd1e1b-1192-41e8-b0da-4d73035de127 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:46.703978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd1e1b-1192-41e8-b0da-4d73035de127 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:46.704397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd1e1b-1192-41e8-b0da-4d73035de127 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:46.705430 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd1e1b-1192-41e8-b0da-4d73035de127 None admin] RBAC: Adding query filter params (interface=public, service_id=1439954d5c5b433bab4ff3225517f49c, region_id=RegionOne) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:46.705886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd1e1b-1192-41e8-b0da-4d73035de127 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:46.707863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd1e1b-1192-41e8-b0da-4d73035de127 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:46.715199 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 458/898] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Sep 21 16:09:46 2022] GET /identity/v3/endpoints?service_id=1439954d5c5b433bab4ff3225517f49c&interface=public®ion_id=RegionOne => generated 194 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:46.719688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd1e1b-1192-41e8-b0da-4d73035de127 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:46.763985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:46.850406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8059b29-bed2-4758-988a-a27e9c1cd4ff None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:46.886688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd1e1b-1192-41e8-b0da-4d73035de127 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:46.899503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:46.912565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55b24c-fdda-4ede-a148-03584c4fecaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:46.913546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55b24c-fdda-4ede-a148-03584c4fecaa None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:46.913961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55b24c-fdda-4ede-a148-03584c4fecaa None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:46.914421 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55b24c-fdda-4ede-a148-03584c4fecaa None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:46.915448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55b24c-fdda-4ede-a148-03584c4fecaa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:46.915877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55b24c-fdda-4ede-a148-03584c4fecaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:46.917409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55b24c-fdda-4ede-a148-03584c4fecaa None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:46.923699 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 441/899] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:09:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:46.927659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55b24c-fdda-4ede-a148-03584c4fecaa None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:46.941786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055de17c-178d-4b5c-b32d-01c76e03846a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:46.942446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055de17c-178d-4b5c-b32d-01c76e03846a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:46.942730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055de17c-178d-4b5c-b32d-01c76e03846a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:46.943548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055de17c-178d-4b5c-b32d-01c76e03846a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:47.015808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055de17c-178d-4b5c-b32d-01c76e03846a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:47.017256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055de17c-178d-4b5c-b32d-01c76e03846a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:47.089053 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55b24c-fdda-4ede-a148-03584c4fecaa None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:47.114993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:47.142487 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 459/900] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:09:46 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:09:47.143394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055de17c-178d-4b5c-b32d-01c76e03846a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:47.149757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea6b98-de9e-4ebb-ab58-68878ac192d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:47.151095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea6b98-de9e-4ebb-ab58-68878ac192d6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:47.151283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea6b98-de9e-4ebb-ab58-68878ac192d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:47.151673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea6b98-de9e-4ebb-ab58-68878ac192d6 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:47.152370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea6b98-de9e-4ebb-ab58-68878ac192d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:47.153844 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea6b98-de9e-4ebb-ab58-68878ac192d6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:47.160605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:47.161632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 442/901] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Sep 21 16:09:46 2022] GET /identity/v3/services => generated 1268 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:09:47.163843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e966725-a5f1-456c-9334-972bae0319ce None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:47.164303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e966725-a5f1-456c-9334-972bae0319ce None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:47.164745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e966725-a5f1-456c-9334-972bae0319ce None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:47.169897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 443/902] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:47.169897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e966725-a5f1-456c-9334-972bae0319ce None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:47.300778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055de17c-178d-4b5c-b32d-01c76e03846a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:47.330442 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e966725-a5f1-456c-9334-972bae0319ce None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:47.341788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921cc75-c450-4e1e-ad24-62b84812b4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:47.342447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921cc75-c450-4e1e-ad24-62b84812b4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:47.342729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921cc75-c450-4e1e-ad24-62b84812b4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:47.343012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921cc75-c450-4e1e-ad24-62b84812b4c4 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:47.343677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921cc75-c450-4e1e-ad24-62b84812b4c4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:47.343982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921cc75-c450-4e1e-ad24-62b84812b4c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:47.345387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b921cc75-c450-4e1e-ad24-62b84812b4c4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:47.358904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 460/903] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:09:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:47.358904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95861114-f9dd-4201-8e2e-294c81148c8c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:47.358904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95861114-f9dd-4201-8e2e-294c81148c8c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:47.358904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95861114-f9dd-4201-8e2e-294c81148c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:47.377779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd97d0-1ba5-4687-a252-c82d7eb4b4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:47.378524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd97d0-1ba5-4687-a252-c82d7eb4b4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:47.378873 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd97d0-1ba5-4687-a252-c82d7eb4b4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:47.379161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd97d0-1ba5-4687-a252-c82d7eb4b4c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:47.384445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-95861114-f9dd-4201-8e2e-294c81148c8c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:47.385471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-95861114-f9dd-4201-8e2e-294c81148c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:47.397383 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd97d0-1ba5-4687-a252-c82d7eb4b4c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:47.399018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd97d0-1ba5-4687-a252-c82d7eb4b4c1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c5dd97d0-1ba5-4687-a252-c82d7eb4b4c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:47.424945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:47.427052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:47.429582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:47.429582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:47.429582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:47.429582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 444/904] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:47.429582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd97d0-1ba5-4687-a252-c82d7eb4b4c1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:47.478996 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:47.499820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 461/905] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:47.502252 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95861114-f9dd-4201-8e2e-294c81148c8c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:47.513874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:47.610916 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd97d0-1ba5-4687-a252-c82d7eb4b4c1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:47.677611 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ff18e-7999-442f-ab98-5c0e97cb236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:47.678727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ff18e-7999-442f-ab98-5c0e97cb236c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:47.679243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ff18e-7999-442f-ab98-5c0e97cb236c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:47.679746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ff18e-7999-442f-ab98-5c0e97cb236c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:47.681075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ff18e-7999-442f-ab98-5c0e97cb236c None admin] RBAC: Adding query filter params (interface=public, service_id=ec8006a59de549bfb211427a724ab5bf, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:47.681650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ff18e-7999-442f-ab98-5c0e97cb236c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:47.684467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ff18e-7999-442f-ab98-5c0e97cb236c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:47.700724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 445/906] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Sep 21 16:09:47 2022] GET /identity/v3/endpoints?service_id=ec8006a59de549bfb211427a724ab5bf&interface=public®ion_id=RegionOne => generated 194 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:47.711558 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ff18e-7999-442f-ab98-5c0e97cb236c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:47.743113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95861114-f9dd-4201-8e2e-294c81148c8c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:47.806817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e8695-2884-42ea-b7d6-8c31510e7d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:47.808179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e8695-2884-42ea-b7d6-8c31510e7d64 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:47.808882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e8695-2884-42ea-b7d6-8c31510e7d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:47.809572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e8695-2884-42ea-b7d6-8c31510e7d64 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:47.810971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e8695-2884-42ea-b7d6-8c31510e7d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:47.811723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e8695-2884-42ea-b7d6-8c31510e7d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:47.814428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e8695-2884-42ea-b7d6-8c31510e7d64 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:47.837372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 462/907] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:47.847993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e8695-2884-42ea-b7d6-8c31510e7d64 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:47.895491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:47.908196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ff18e-7999-442f-ab98-5c0e97cb236c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:47.968864 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:47.969758 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f0bd1-8994-4bd7-9779-52d0e3af6920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:47.971055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f0bd1-8994-4bd7-9779-52d0e3af6920 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:47.971767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f0bd1-8994-4bd7-9779-52d0e3af6920 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:47.972511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f0bd1-8994-4bd7-9779-52d0e3af6920 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:47.973705 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f0bd1-8994-4bd7-9779-52d0e3af6920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:47.976365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f0bd1-8994-4bd7-9779-52d0e3af6920 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:47.987611 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 446/908] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Sep 21 16:09:47 2022] GET /identity/v3/services => generated 1268 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:09:48.041344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f0bd1-8994-4bd7-9779-52d0e3af6920 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:48.053388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e8695-2884-42ea-b7d6-8c31510e7d64 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:48.127250 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fc2bd-1c3c-4c78-a6c9-ca94ad208be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:48.128570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fc2bd-1c3c-4c78-a6c9-ca94ad208be1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:48.129044 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fc2bd-1c3c-4c78-a6c9-ca94ad208be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:48.129527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fc2bd-1c3c-4c78-a6c9-ca94ad208be1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:48.143058 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fc2bd-1c3c-4c78-a6c9-ca94ad208be1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:48.145381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fc2bd-1c3c-4c78-a6c9-ca94ad208be1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a91fc2bd-1c3c-4c78-a6c9-ca94ad208be1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:48.171584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:48.172561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:48.173537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:48.173537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:48.173537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:48.173537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:48.173537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:48.173537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:48.173537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:48.173537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:48.180075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 463/909] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:09:48.185369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fc2bd-1c3c-4c78-a6c9-ca94ad208be1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:48.240138 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f0bd1-8994-4bd7-9779-52d0e3af6920 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:48.302385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d70de-3e67-4a7c-90db-6c2429bd40df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:48.303322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d70de-3e67-4a7c-90db-6c2429bd40df None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:48.303801 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d70de-3e67-4a7c-90db-6c2429bd40df None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:48.305304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d70de-3e67-4a7c-90db-6c2429bd40df None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:48.306503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d70de-3e67-4a7c-90db-6c2429bd40df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:48.308932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d70de-3e67-4a7c-90db-6c2429bd40df None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:48.334648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 447/910] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:09:48 2022] POST /identity/v3/services => generated 241 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:09:48.388865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fc2bd-1c3c-4c78-a6c9-ca94ad208be1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:48.434427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:48.465117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff659d66-1c57-4ea0-9e84-e29d1a8865f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:48.466225 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff659d66-1c57-4ea0-9e84-e29d1a8865f2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:48.466647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff659d66-1c57-4ea0-9e84-e29d1a8865f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:48.467063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff659d66-1c57-4ea0-9e84-e29d1a8865f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:48.468196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff659d66-1c57-4ea0-9e84-e29d1a8865f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:48.468797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff659d66-1c57-4ea0-9e84-e29d1a8865f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:48.471369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff659d66-1c57-4ea0-9e84-e29d1a8865f2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:48.504981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 464/911] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:48.515351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d70de-3e67-4a7c-90db-6c2429bd40df None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:48.552015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:48.717491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d70de-3e67-4a7c-90db-6c2429bd40df None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:48.781197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8f077-0dfb-4806-beed-23cdcc9277b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:48.782129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f077-0dfb-4806-beed-23cdcc9277b8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:48.782619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f077-0dfb-4806-beed-23cdcc9277b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:48.783108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f077-0dfb-4806-beed-23cdcc9277b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:48.784517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8f077-0dfb-4806-beed-23cdcc9277b8 None admin] RBAC: Adding query filter params (role.id=8e59f99483d148c09246abf67508a11b, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=8da8d12ac74644f280ba31da1715f603) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:48.785064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8f077-0dfb-4806-beed-23cdcc9277b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:48.788415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8f077-0dfb-4806-beed-23cdcc9277b8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:48.799768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 448/912] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:09:48 2022] GET /identity/v3/role_assignments?user.id=8da8d12ac74644f280ba31da1715f603&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=8e59f99483d148c09246abf67508a11b => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:50.238795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678f23-2f4c-4b6f-a292-8e8f9f4dbe8e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:50.240984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678f23-2f4c-4b6f-a292-8e8f9f4dbe8e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:50.241785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678f23-2f4c-4b6f-a292-8e8f9f4dbe8e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:50.244875 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 465/913] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:50.254395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4866dd6-80a2-40bc-bd22-280e2fe0d931 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:50.256400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4866dd6-80a2-40bc-bd22-280e2fe0d931 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:50.257176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4866dd6-80a2-40bc-bd22-280e2fe0d931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:50.287933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a4866dd6-80a2-40bc-bd22-280e2fe0d931 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:50.290532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a4866dd6-80a2-40bc-bd22-280e2fe0d931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:50.421343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 449/914] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:50.931921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72678f23-2f4c-4b6f-a292-8e8f9f4dbe8e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:51.143962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72678f23-2f4c-4b6f-a292-8e8f9f4dbe8e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:51.183059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc439d-e2cc-4561-a605-2585bdd6a439 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:51.183479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc439d-e2cc-4561-a605-2585bdd6a439 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:51.184025 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc439d-e2cc-4561-a605-2585bdd6a439 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:51.193755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 450/915] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:09:51.196191 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bcc04-14fb-4399-9d9e-5d4ae613a3c8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:51.196694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bcc04-14fb-4399-9d9e-5d4ae613a3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:51.197281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbed2bc-0c96-44a2-8fc5-45603371d65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:51.197476 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bcc04-14fb-4399-9d9e-5d4ae613a3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:51.198540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbed2bc-0c96-44a2-8fc5-45603371d65b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:51.199045 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbed2bc-0c96-44a2-8fc5-45603371d65b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:51.199529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbed2bc-0c96-44a2-8fc5-45603371d65b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:51.200336 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbed2bc-0c96-44a2-8fc5-45603371d65b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:51.201984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbed2bc-0c96-44a2-8fc5-45603371d65b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-7dbed2bc-0c96-44a2-8fc5-45603371d65b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:51.219960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:51.221060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 21 16:09:51.222134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:51.224650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 466/916] 10.222.0.64 () {56 vars in 1132 bytes} [Wed Sep 21 16:09:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:51.230737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbed2bc-0c96-44a2-8fc5-45603371d65b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:51.237962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-918bcc04-14fb-4399-9d9e-5d4ae613a3c8 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:51.240688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-918bcc04-14fb-4399-9d9e-5d4ae613a3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:51.254952 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:51.283914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:51.354769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbed2bc-0c96-44a2-8fc5-45603371d65b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:51.382411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 451/917] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:51.391635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c3548-f0d4-4241-a33c-9afedc277a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:51.392449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c3548-f0d4-4241-a33c-9afedc277a01 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:51.392869 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c3548-f0d4-4241-a33c-9afedc277a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:51.393282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c3548-f0d4-4241-a33c-9afedc277a01 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:51.394076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c3548-f0d4-4241-a33c-9afedc277a01 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:51.394542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c3548-f0d4-4241-a33c-9afedc277a01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:51.396604 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c3548-f0d4-4241-a33c-9afedc277a01 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:51.402387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 467/918] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Sep 21 16:09:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:51.408971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bcc04-14fb-4399-9d9e-5d4ae613a3c8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:51.437814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a40d3-1b05-49de-a56f-5350cf4c6dda None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:51.438257 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a40d3-1b05-49de-a56f-5350cf4c6dda None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:51.438725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511a40d3-1b05-49de-a56f-5350cf4c6dda None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:51.440808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 468/919] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:51.447460 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9f1c5-c28d-405e-82fb-3b58a6234375 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:51.447728 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9f1c5-c28d-405e-82fb-3b58a6234375 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:51.447999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9f1c5-c28d-405e-82fb-3b58a6234375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:51.468471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9ee9f1c5-c28d-405e-82fb-3b58a6234375 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:51.469770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9ee9f1c5-c28d-405e-82fb-3b58a6234375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:51.558948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bcc04-14fb-4399-9d9e-5d4ae613a3c8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:51.581274 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 469/920] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:09:51.624965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acd7fe-3200-4879-98f5-b1cecdb8db96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:51.625981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acd7fe-3200-4879-98f5-b1cecdb8db96 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:51.626503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acd7fe-3200-4879-98f5-b1cecdb8db96 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:51.626977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acd7fe-3200-4879-98f5-b1cecdb8db96 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:51.628091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acd7fe-3200-4879-98f5-b1cecdb8db96 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:51.628637 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acd7fe-3200-4879-98f5-b1cecdb8db96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:51.631761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0acd7fe-3200-4879-98f5-b1cecdb8db96 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:51.641384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 452/921] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Sep 21 16:09:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:51.647497 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9f1c5-c28d-405e-82fb-3b58a6234375 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:51.712835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:51.729090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acd7fe-3200-4879-98f5-b1cecdb8db96 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:51.739636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:51.841876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9f1c5-c28d-405e-82fb-3b58a6234375 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:51.848055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acd7fe-3200-4879-98f5-b1cecdb8db96 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:51.886166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9824b6-f258-423f-9017-7a193ed55ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:51.886796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9824b6-f258-423f-9017-7a193ed55ffe None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:51.887072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9824b6-f258-423f-9017-7a193ed55ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:51.887344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9824b6-f258-423f-9017-7a193ed55ffe None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:51.888036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9824b6-f258-423f-9017-7a193ed55ffe None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:51.889392 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9824b6-f258-423f-9017-7a193ed55ffe None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:51.891433 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcaaa2-2ef4-428f-89c1-4b14f4993234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:51.892063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcaaa2-2ef4-428f-89c1-4b14f4993234 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:51.892336 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcaaa2-2ef4-428f-89c1-4b14f4993234 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:51.892611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfcaaa2-2ef4-428f-89c1-4b14f4993234 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:51.893296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcaaa2-2ef4-428f-89c1-4b14f4993234 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:51.894697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfcaaa2-2ef4-428f-89c1-4b14f4993234 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-8a9824b6-f258-423f-9017-7a193ed55ffe None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:51.899074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:51.901030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:51.903216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 453/922] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:09:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:51.904871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21c9b5-e756-420b-9304-2e8a3c6d202d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:51.904871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21c9b5-e756-420b-9304-2e8a3c6d202d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:51.904871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f21c9b5-e756-420b-9304-2e8a3c6d202d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:51.905687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 454/923] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:51.910462 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21c9b5-e756-420b-9304-2e8a3c6d202d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:51.926657 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 470/924] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:09:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:09:51.927977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcaaa2-2ef4-428f-89c1-4b14f4993234 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:52.014241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:52.068853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:52.075045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f21c9b5-e756-420b-9304-2e8a3c6d202d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:52.088815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfcaaa2-2ef4-428f-89c1-4b14f4993234 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:52.134246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6bb03-136f-4477-9551-cae7402b86e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:52.135238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6bb03-136f-4477-9551-cae7402b86e7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:52.135687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6bb03-136f-4477-9551-cae7402b86e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:52.136094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6bb03-136f-4477-9551-cae7402b86e7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:52.147336 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4923e8f-116e-4a7c-bade-443e398a318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:52.148363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4923e8f-116e-4a7c-bade-443e398a318d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:52.148787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4923e8f-116e-4a7c-bade-443e398a318d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:52.149376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4923e8f-116e-4a7c-bade-443e398a318d None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:52.150256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6bb03-136f-4477-9551-cae7402b86e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:52.150619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4923e8f-116e-4a7c-bade-443e398a318d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:52.151064 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4923e8f-116e-4a7c-bade-443e398a318d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:52.152963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4923e8f-116e-4a7c-bade-443e398a318d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-bbc6bb03-136f-4477-9551-cae7402b86e7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:52.155290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:52.156918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:52.158414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 455/925] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Sep 21 16:09:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:52.159285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc11c01-7590-44c2-abeb-41257b8ff09f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:52.159285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc11c01-7590-44c2-abeb-41257b8ff09f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:52.159285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc11c01-7590-44c2-abeb-41257b8ff09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:52.165038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 471/926] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:09:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:52.165964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4923e8f-116e-4a7c-bade-443e398a318d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:52.183205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-edc11c01-7590-44c2-abeb-41257b8ff09f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:52.184402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-edc11c01-7590-44c2-abeb-41257b8ff09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:52.283247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 456/927] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:52.285719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc11c01-7590-44c2-abeb-41257b8ff09f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:52.311505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4923e8f-116e-4a7c-bade-443e398a318d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:52.395632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d23124-68b6-4fe1-adfb-729961ea9ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:52.398096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23124-68b6-4fe1-adfb-729961ea9ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:52.398924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23124-68b6-4fe1-adfb-729961ea9ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:52.399682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d23124-68b6-4fe1-adfb-729961ea9ad4 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:52.401247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d23124-68b6-4fe1-adfb-729961ea9ad4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:52.402309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d23124-68b6-4fe1-adfb-729961ea9ad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:52.403597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:52.405572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d23124-68b6-4fe1-adfb-729961ea9ad4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:52.429072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 472/928] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Sep 21 16:09:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:52.432780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d23124-68b6-4fe1-adfb-729961ea9ad4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:52.471013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:52.511193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc11c01-7590-44c2-abeb-41257b8ff09f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:52.584249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755498f-4be7-4e77-81a9-e0077ee12541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:52.585767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755498f-4be7-4e77-81a9-e0077ee12541 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:52.586583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755498f-4be7-4e77-81a9-e0077ee12541 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:52.587363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2755498f-4be7-4e77-81a9-e0077ee12541 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:52.588805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755498f-4be7-4e77-81a9-e0077ee12541 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:52.589636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755498f-4be7-4e77-81a9-e0077ee12541 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:52.592483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2755498f-4be7-4e77-81a9-e0077ee12541 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:52.608644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 457/929] 10.222.0.64 () {56 vars in 1114 bytes} [Wed Sep 21 16:09:52 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:52.612911 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755498f-4be7-4e77-81a9-e0077ee12541 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:52.651444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d23124-68b6-4fe1-adfb-729961ea9ad4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:52.708513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e3221-0fd2-457b-a4b0-aad39782077e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:52.710205 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e3221-0fd2-457b-a4b0-aad39782077e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:52.710637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e3221-0fd2-457b-a4b0-aad39782077e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:52.711070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e3221-0fd2-457b-a4b0-aad39782077e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:52.726627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e3221-0fd2-457b-a4b0-aad39782077e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:52.728592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e3221-0fd2-457b-a4b0-aad39782077e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-fe4e3221-0fd2-457b-a4b0-aad39782077e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:52.753251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:52.754400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:52.760945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:52.760945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:52.760945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:52.760945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 473/930] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:09:52.760945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e3221-0fd2-457b-a4b0-aad39782077e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:52.784637 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2755498f-4be7-4e77-81a9-e0077ee12541 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:52.818324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:52.843266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7bfcd-451a-449f-9a65-55221cf0d038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:52.843771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7bfcd-451a-449f-9a65-55221cf0d038 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:52.844155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7bfcd-451a-449f-9a65-55221cf0d038 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:52.844430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7bfcd-451a-449f-9a65-55221cf0d038 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:52.845103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc7bfcd-451a-449f-9a65-55221cf0d038 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:52.846529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc7bfcd-451a-449f-9a65-55221cf0d038 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:52.869236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:52.879481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 458/931] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:09:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:09:52.882497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7bfcd-451a-449f-9a65-55221cf0d038 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:52.926510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e3221-0fd2-457b-a4b0-aad39782077e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:53.011959 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b879a8-e6ce-4afa-b98f-277d3df79c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:53.013698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b879a8-e6ce-4afa-b98f-277d3df79c22 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:53.014558 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b879a8-e6ce-4afa-b98f-277d3df79c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:53.015185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b879a8-e6ce-4afa-b98f-277d3df79c22 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:53.016441 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b879a8-e6ce-4afa-b98f-277d3df79c22 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:53.019522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b879a8-e6ce-4afa-b98f-277d3df79c22 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-79b879a8-e6ce-4afa-b98f-277d3df79c22 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:53.034013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:53.035117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:09:53.035966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:53.039428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 474/932] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Sep 21 16:09:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:53.047508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b879a8-e6ce-4afa-b98f-277d3df79c22 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:53.049719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7bfcd-451a-449f-9a65-55221cf0d038 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:53.131083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a0f56-58d4-4c8a-98d5-9cafbf0a2be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:53.133585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a0f56-58d4-4c8a-98d5-9cafbf0a2be7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:53.133585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a0f56-58d4-4c8a-98d5-9cafbf0a2be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:53.133585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a0f56-58d4-4c8a-98d5-9cafbf0a2be7 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:53.134312 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a0f56-58d4-4c8a-98d5-9cafbf0a2be7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:53.134860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a0f56-58d4-4c8a-98d5-9cafbf0a2be7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:53.137580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5a0f56-58d4-4c8a-98d5-9cafbf0a2be7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:53.160865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 459/933] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:53.169950 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a0f56-58d4-4c8a-98d5-9cafbf0a2be7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:53.210826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b879a8-e6ce-4afa-b98f-277d3df79c22 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:53.230410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:53.269041 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:53.287283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9d469-d29f-442d-a09a-308c9408a157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:53.288852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9d469-d29f-442d-a09a-308c9408a157 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:53.290378 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9d469-d29f-442d-a09a-308c9408a157 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:53.290660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9d469-d29f-442d-a09a-308c9408a157 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:53.291336 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9d469-d29f-442d-a09a-308c9408a157 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:53.294017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9d469-d29f-442d-a09a-308c9408a157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:53.295406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9d469-d29f-442d-a09a-308c9408a157 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:53.306231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 475/934] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Sep 21 16:09:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:53.311068 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9d469-d29f-442d-a09a-308c9408a157 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:53.328508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a0f56-58d4-4c8a-98d5-9cafbf0a2be7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:53.371182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3b94f-4792-4c3f-b326-d48d8f6e5b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:53.372389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b94f-4792-4c3f-b326-d48d8f6e5b6a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:53.372675 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b94f-4792-4c3f-b326-d48d8f6e5b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:53.372949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3b94f-4792-4c3f-b326-d48d8f6e5b6a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:53.381925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3b94f-4792-4c3f-b326-d48d8f6e5b6a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:53.383331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3b94f-4792-4c3f-b326-d48d8f6e5b6a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-82d3b94f-4792-4c3f-b326-d48d8f6e5b6a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:09:53.397533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:53.398563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:53.399474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:53.399474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:53.399474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:09:53.399474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:09:53.399474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:09:53.399474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:09:53.399474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:09:53.399474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:09:53.399474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 16:09:53.399474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:53.399992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 460/935] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Sep 21 16:09:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:09:53.406004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3b94f-4792-4c3f-b326-d48d8f6e5b6a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:53.454337 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9d469-d29f-442d-a09a-308c9408a157 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:53.516491 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1d167-b98b-470d-bf34-9a9b5f5488c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:53.517474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1d167-b98b-470d-bf34-9a9b5f5488c0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:53.517909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1d167-b98b-470d-bf34-9a9b5f5488c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:53.518309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1d167-b98b-470d-bf34-9a9b5f5488c0 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:53.519276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1d167-b98b-470d-bf34-9a9b5f5488c0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:53.519911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1d167-b98b-470d-bf34-9a9b5f5488c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:53.521830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1d167-b98b-470d-bf34-9a9b5f5488c0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:53.534553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 476/936] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Sep 21 16:09:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:53.536698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1d167-b98b-470d-bf34-9a9b5f5488c0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:53.561953 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:53.570692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3b94f-4792-4c3f-b326-d48d8f6e5b6a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:53.613571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf49fd-7673-4cb3-908d-911ea8a06826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:53.615041 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf49fd-7673-4cb3-908d-911ea8a06826 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:53.615567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf49fd-7673-4cb3-908d-911ea8a06826 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:53.615984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf49fd-7673-4cb3-908d-911ea8a06826 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:53.616877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf49fd-7673-4cb3-908d-911ea8a06826 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:53.617331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf49fd-7673-4cb3-908d-911ea8a06826 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:53.618958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf49fd-7673-4cb3-908d-911ea8a06826 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:53.630381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:53.644569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 461/937] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Sep 21 16:09:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:53.654449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf49fd-7673-4cb3-908d-911ea8a06826 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:53.708900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1d167-b98b-470d-bf34-9a9b5f5488c0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:53.755926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935b374-61ea-48bd-845c-bade5c4bd6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:53.756551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935b374-61ea-48bd-845c-bade5c4bd6fe None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:53.756858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935b374-61ea-48bd-845c-bade5c4bd6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:53.757087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935b374-61ea-48bd-845c-bade5c4bd6fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:53.757804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935b374-61ea-48bd-845c-bade5c4bd6fe None admin] RBAC: Adding query filter params (interface=public, service_id=cd1e5d1c3cbf42a78892a179dad103bb, region_id=RegionOne) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:53.758140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935b374-61ea-48bd-845c-bade5c4bd6fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:53.759504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c935b374-61ea-48bd-845c-bade5c4bd6fe None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:53.770296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 477/938] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Sep 21 16:09:53 2022] GET /identity/v3/endpoints?service_id=cd1e5d1c3cbf42a78892a179dad103bb&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:53.771369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935b374-61ea-48bd-845c-bade5c4bd6fe None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:53.806519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf49fd-7673-4cb3-908d-911ea8a06826 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:53.857419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b02c10-5fce-49a0-8ecc-400af48fa31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:53.858283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b02c10-5fce-49a0-8ecc-400af48fa31f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:53.858631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b02c10-5fce-49a0-8ecc-400af48fa31f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:53.859044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b02c10-5fce-49a0-8ecc-400af48fa31f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:53.873012 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b02c10-5fce-49a0-8ecc-400af48fa31f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:53.874995 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b02c10-5fce-49a0-8ecc-400af48fa31f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-46b02c10-5fce-49a0-8ecc-400af48fa31f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:53.895853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:53.896858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:53.897803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:53.897803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:53.897803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:53.902394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 462/939] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:09:53.906915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b02c10-5fce-49a0-8ecc-400af48fa31f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:53.921301 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:53.932359 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935b374-61ea-48bd-845c-bade5c4bd6fe None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:53.975771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:53.996963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b530a-396b-47a3-a758-c903368077f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:53.998290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b530a-396b-47a3-a758-c903368077f6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:53.998651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b530a-396b-47a3-a758-c903368077f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:53.999197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b530a-396b-47a3-a758-c903368077f6 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:54.000527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b530a-396b-47a3-a758-c903368077f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:54.003334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b530a-396b-47a3-a758-c903368077f6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:54.022350 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 478/940] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Sep 21 16:09:53 2022] GET /identity/v3/services => generated 1497 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:09:54.044481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b02c10-5fce-49a0-8ecc-400af48fa31f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:54.088952 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a51101-cd5c-4214-ad32-09e09b1d4a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:54.089920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a51101-cd5c-4214-ad32-09e09b1d4a18 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:54.090421 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a51101-cd5c-4214-ad32-09e09b1d4a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:54.090973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a51101-cd5c-4214-ad32-09e09b1d4a18 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:54.091866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a51101-cd5c-4214-ad32-09e09b1d4a18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:54.092384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a51101-cd5c-4214-ad32-09e09b1d4a18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:54.094031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a51101-cd5c-4214-ad32-09e09b1d4a18 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:54.109779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 463/941] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:54.118492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b530a-396b-47a3-a758-c903368077f6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:54.314957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b530a-396b-47a3-a758-c903368077f6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:54.353716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74800c4-47eb-4ddf-866e-39d6545437bb None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:54.354472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74800c4-47eb-4ddf-866e-39d6545437bb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:54.354472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74800c4-47eb-4ddf-866e-39d6545437bb None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:54.356061 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 464/942] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:09:54.367064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca060caa-3001-48ff-8e87-67c2fc0257b9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:54.367365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca060caa-3001-48ff-8e87-67c2fc0257b9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:54.367629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca060caa-3001-48ff-8e87-67c2fc0257b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:54.370875 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b96a5-5506-4d8f-8378-a6da47cbf2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:54.371988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b96a5-5506-4d8f-8378-a6da47cbf2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:54.372443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b96a5-5506-4d8f-8378-a6da47cbf2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:54.372912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b96a5-5506-4d8f-8378-a6da47cbf2c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:54.382190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b96a5-5506-4d8f-8378-a6da47cbf2c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:54.384149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b96a5-5506-4d8f-8378-a6da47cbf2c2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:54.390374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ca060caa-3001-48ff-8e87-67c2fc0257b9 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:54.391741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ca060caa-3001-48ff-8e87-67c2fc0257b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-de5b96a5-5506-4d8f-8378-a6da47cbf2c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:54.403842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:54.405529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:54.406859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:54.406859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:54.406859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:54.406859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:54.406859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:54.406859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:54.406859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:54.406859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:54.409311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 479/943] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:54.416606 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b96a5-5506-4d8f-8378-a6da47cbf2c2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:54.493685 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:54.511891 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 465/944] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:09:54.523342 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:54.621587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b96a5-5506-4d8f-8378-a6da47cbf2c2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:54.693295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e8a55-7f75-42ea-beb1-b9ae4614ec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:54.694701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e8a55-7f75-42ea-beb1-b9ae4614ec93 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:54.695408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e8a55-7f75-42ea-beb1-b9ae4614ec93 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:54.696111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e8a55-7f75-42ea-beb1-b9ae4614ec93 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:54.698116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e8a55-7f75-42ea-beb1-b9ae4614ec93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:54.699003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e8a55-7f75-42ea-beb1-b9ae4614ec93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:54.702070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e8a55-7f75-42ea-beb1-b9ae4614ec93 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:54.728400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 480/945] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:09:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:54.738667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca060caa-3001-48ff-8e87-67c2fc0257b9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:54.828842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e8a55-7f75-42ea-beb1-b9ae4614ec93 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:54.953206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e8a55-7f75-42ea-beb1-b9ae4614ec93 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:54.963222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:54.980590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca060caa-3001-48ff-8e87-67c2fc0257b9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:55.014525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58cc36-08d6-45ef-b8cf-4f7e5416fd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:55.015675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58cc36-08d6-45ef-b8cf-4f7e5416fd31 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:55.016162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58cc36-08d6-45ef-b8cf-4f7e5416fd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:55.016637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58cc36-08d6-45ef-b8cf-4f7e5416fd31 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:55.017439 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58cc36-08d6-45ef-b8cf-4f7e5416fd31 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:55.019214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58cc36-08d6-45ef-b8cf-4f7e5416fd31 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4e58cc36-08d6-45ef-b8cf-4f7e5416fd31 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:55.030614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:55.031686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 16:09:55.032535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:55.035062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50fde7-c2bd-4517-807e-6799ec10ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:55.040793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50fde7-c2bd-4517-807e-6799ec10ccb3 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:55.040793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50fde7-c2bd-4517-807e-6799ec10ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:55.040793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50fde7-c2bd-4517-807e-6799ec10ccb3 None admin] PATH_INFO: `/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/8da8d12ac74644f280ba31da1715f603/roles/8e59f99483d148c09246abf67508a11b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:55.041582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 481/946] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:09:55.042162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50fde7-c2bd-4517-807e-6799ec10ccb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9e1d9331b4d4dbd3dd447937c42ad, user_id=8da8d12ac74644f280ba31da1715f603, role_id=8e59f99483d148c09246abf67508a11b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:55.042999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58cc36-08d6-45ef-b8cf-4f7e5416fd31 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:55.045004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50fde7-c2bd-4517-807e-6799ec10ccb3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:55.076099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 466/947] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:09:54 2022] PUT /identity/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/8da8d12ac74644f280ba31da1715f603/roles/8e59f99483d148c09246abf67508a11b => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:09:55.261394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58cc36-08d6-45ef-b8cf-4f7e5416fd31 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:55.328745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd56d5-960c-4d58-919b-dc9515fad1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:55.329743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd56d5-960c-4d58-919b-dc9515fad1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:55.330180 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd56d5-960c-4d58-919b-dc9515fad1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:55.330578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd56d5-960c-4d58-919b-dc9515fad1d9 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:55.331587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd56d5-960c-4d58-919b-dc9515fad1d9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:55.332030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd56d5-960c-4d58-919b-dc9515fad1d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:55.334256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd56d5-960c-4d58-919b-dc9515fad1d9 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:55.342357 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 482/948] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:09:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:55.349691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50fde7-c2bd-4517-807e-6799ec10ccb3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:55.463319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:55.478458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928569a7-f95f-4476-9491-74635a911285 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:55.478791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928569a7-f95f-4476-9491-74635a911285 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:55.479042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928569a7-f95f-4476-9491-74635a911285 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:55.483369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 483/949] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:09:55.487506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366b810-3de1-490f-8482-ffe4b0d388e9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:55.488002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366b810-3de1-490f-8482-ffe4b0d388e9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:55.488507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366b810-3de1-490f-8482-ffe4b0d388e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:55.525375 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50fde7-c2bd-4517-807e-6799ec10ccb3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:55.530521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f366b810-3de1-490f-8482-ffe4b0d388e9 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:55.532694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f366b810-3de1-490f-8482-ffe4b0d388e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:55.572032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534a0fe-b69b-4d72-8cae-cd80402c6b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:55.573317 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534a0fe-b69b-4d72-8cae-cd80402c6b2d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:55.573809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534a0fe-b69b-4d72-8cae-cd80402c6b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:55.574829 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534a0fe-b69b-4d72-8cae-cd80402c6b2d None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:55.576310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534a0fe-b69b-4d72-8cae-cd80402c6b2d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:55.576851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534a0fe-b69b-4d72-8cae-cd80402c6b2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:55.578120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:55.579597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534a0fe-b69b-4d72-8cae-cd80402c6b2d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:55.597644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 467/950] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:09:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:55.663726 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 484/951] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:55.943828 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534a0fe-b69b-4d72-8cae-cd80402c6b2d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:56.140773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534a0fe-b69b-4d72-8cae-cd80402c6b2d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:56.180825 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:56.212882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2795de2f-e934-4b4e-9f47-2a9b22a5a892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:56.214432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795de2f-e934-4b4e-9f47-2a9b22a5a892 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:56.215221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795de2f-e934-4b4e-9f47-2a9b22a5a892 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:56.215974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2795de2f-e934-4b4e-9f47-2a9b22a5a892 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:56.217543 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795de2f-e934-4b4e-9f47-2a9b22a5a892 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:56.220384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2795de2f-e934-4b4e-9f47-2a9b22a5a892 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:56.225966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: WARNING keystone.server.flask.common [None req-2795de2f-e934-4b4e-9f47-2a9b22a5a892 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 21 16:09:56.260381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.sql.core [None req-2795de2f-e934-4b4e-9f47-2a9b22a5a892 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 21 16:09:56.260381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 21 16:09:56.260381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [parameters: {'id': '688331fd18744f8a8600479bb8a09ff3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 21 16:09:56.260381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81425) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-2795de2f-e934-4b4e-9f47-2a9b22a5a892 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 688331fd18744f8a8600479bb8a09ff3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 688331fd18744f8a8600479bb8a09ff3. Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application conn.query(q) Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 16:09:56.278429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result.read() Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 21 16:09:56.280052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application next(self.gen) Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application yield resource Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application next(self.gen) Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.session.rollback() Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.force_reraise() Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise self.value Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application session.commit() Sep 21 16:09:56.282835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._prepare_impl() Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.session.flush() Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._flush(objects) Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application compat.raise_( Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application flush_context.execute() Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 21 16:09:56.284860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rec.execute(self) Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 21 16:09:56.287729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application conn.query(q) Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result.read() Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 16:09:56.298668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [parameters: {'id': '688331fd18744f8a8600479bb8a09ff3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:56.302148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:56.305447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:56.316474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:56.316474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 21 16:09:56.316474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 21 16:09:56.316474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:56.316474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:56.316474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 21 16:09:56.316474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 21 16:09:56.316474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 688331fd18744f8a8600479bb8a09ff3. Sep 21 16:09:56.316474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:56.316474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 468/952] 10.222.0.64 () {60 vars in 1140 bytes} [Wed Sep 21 16:09:55 2022] POST /identity/v3/projects => generated 262 bytes in 339 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:56.318540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366b810-3de1-490f-8482-ffe4b0d388e9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:56.411545 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd74f0-7dd1-4866-be82-15342f746a2a None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:56.411840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd74f0-7dd1-4866-be82-15342f746a2a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:56.412173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd74f0-7dd1-4866-be82-15342f746a2a None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:56.421158 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 469/953] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:56.422678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100940e5-8bea-437b-bb61-2f4031f59436 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:56.423029 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100940e5-8bea-437b-bb61-2f4031f59436 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:56.423211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100940e5-8bea-437b-bb61-2f4031f59436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:56.462820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-100940e5-8bea-437b-bb61-2f4031f59436 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:56.467069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-100940e5-8bea-437b-bb61-2f4031f59436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:56.501654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366b810-3de1-490f-8482-ffe4b0d388e9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:56.596700 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39234428-1d1e-4a54-be68-31c74565142e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:56.600711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39234428-1d1e-4a54-be68-31c74565142e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:56.601542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39234428-1d1e-4a54-be68-31c74565142e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:56.602627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39234428-1d1e-4a54-be68-31c74565142e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:56.617692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39234428-1d1e-4a54-be68-31c74565142e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:56.620589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39234428-1d1e-4a54-be68-31c74565142e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:56.622234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 470/954] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-39234428-1d1e-4a54-be68-31c74565142e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:09:56.647117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:56.649027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:56.651019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:09:56.651019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:09:56.651019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:09:56.651019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:09:56.651019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:09:56.651019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:09:56.651019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:09:56.651019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:56.655047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 485/955] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:09:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:56.659097 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100940e5-8bea-437b-bb61-2f4031f59436 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:56.847993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100940e5-8bea-437b-bb61-2f4031f59436 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:56.867049 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:56.890687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39234428-1d1e-4a54-be68-31c74565142e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:56.919773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36967d-4ec2-4121-a737-98d9c589eeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:56.920811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36967d-4ec2-4121-a737-98d9c589eeae None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:56.921279 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36967d-4ec2-4121-a737-98d9c589eeae None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:56.921722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e36967d-4ec2-4121-a737-98d9c589eeae None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:56.921923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:56.922880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36967d-4ec2-4121-a737-98d9c589eeae None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:56.923393 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36967d-4ec2-4121-a737-98d9c589eeae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:56.926633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e36967d-4ec2-4121-a737-98d9c589eeae None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:56.970612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 471/956] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:09:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:57.069952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39234428-1d1e-4a54-be68-31c74565142e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:57.137607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c6821-fab0-4317-b31a-171cadb34df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:57.138197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c6821-fab0-4317-b31a-171cadb34df0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:57.138481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c6821-fab0-4317-b31a-171cadb34df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:57.138754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c6821-fab0-4317-b31a-171cadb34df0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:57.139358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c6821-fab0-4317-b31a-171cadb34df0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:57.140742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027c6821-fab0-4317-b31a-171cadb34df0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-027c6821-fab0-4317-b31a-171cadb34df0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:57.159523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:57.161513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:09:57.163734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:57.168650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 486/957] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Sep 21 16:09:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:09:57.174294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36967d-4ec2-4121-a737-98d9c589eeae None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:57.303661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e36967d-4ec2-4121-a737-98d9c589eeae None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:57.344143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618b046-b4f9-4787-a7c2-4c56477b8066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:57.344763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618b046-b4f9-4787-a7c2-4c56477b8066 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:57.345092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618b046-b4f9-4787-a7c2-4c56477b8066 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:57.345376 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b618b046-b4f9-4787-a7c2-4c56477b8066 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:57.346020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618b046-b4f9-4787-a7c2-4c56477b8066 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:57.346388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618b046-b4f9-4787-a7c2-4c56477b8066 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:57.347851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b618b046-b4f9-4787-a7c2-4c56477b8066 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:57.354416 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 472/958] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Sep 21 16:09:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:57.360959 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c6821-fab0-4317-b31a-171cadb34df0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:57.473608 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:57.498470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c6821-fab0-4317-b31a-171cadb34df0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:57.564553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669777de-4f7c-4680-8ae2-cf98aba40f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:57.565505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669777de-4f7c-4680-8ae2-cf98aba40f7b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:57.565948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669777de-4f7c-4680-8ae2-cf98aba40f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:57.566448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669777de-4f7c-4680-8ae2-cf98aba40f7b None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:57.568081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669777de-4f7c-4680-8ae2-cf98aba40f7b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:57.568634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669777de-4f7c-4680-8ae2-cf98aba40f7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:57.571097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669777de-4f7c-4680-8ae2-cf98aba40f7b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:57.581393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 487/959] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Sep 21 16:09:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:09:57.587363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618b046-b4f9-4787-a7c2-4c56477b8066 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:57.622187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:57.757735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b618b046-b4f9-4787-a7c2-4c56477b8066 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:57.819466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf11385-bbe9-4e32-8618-13f13accad7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:57.820822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf11385-bbe9-4e32-8618-13f13accad7f None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:57.821821 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf11385-bbe9-4e32-8618-13f13accad7f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:57.822548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf11385-bbe9-4e32-8618-13f13accad7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:57.824017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf11385-bbe9-4e32-8618-13f13accad7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:57.826673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf11385-bbe9-4e32-8618-13f13accad7f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:57.868282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 473/960] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:09:57 2022] POST /identity/v3/endpoints => generated 349 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:09:57.988357 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970ffd9-f50a-4c28-8f82-88e60d9b94fc None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:57.989627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970ffd9-f50a-4c28-8f82-88e60d9b94fc None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:57.989905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970ffd9-f50a-4c28-8f82-88e60d9b94fc None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:57.993020 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 488/961] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:57.998792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c30572-ba2c-4530-bc14-da5a091c5ff9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:57.999077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c30572-ba2c-4530-bc14-da5a091c5ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:57.999403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c30572-ba2c-4530-bc14-da5a091c5ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:58.023323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e1c30572-ba2c-4530-bc14-da5a091c5ff9 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:58.025408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e1c30572-ba2c-4530-bc14-da5a091c5ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:58.174831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 474/962] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:58.390136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112c449-3aab-4bce-829d-d7ba44b63c49 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:58.390899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112c449-3aab-4bce-829d-d7ba44b63c49 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:58.391649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b112c449-3aab-4bce-829d-d7ba44b63c49 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:58.394781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 489/963] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:09:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:09:58.402039 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f63f1-b7c0-4b53-a705-f204de847c9c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:58.402583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f63f1-b7c0-4b53-a705-f204de847c9c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:58.403072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f63f1-b7c0-4b53-a705-f204de847c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:58.435221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-861f63f1-b7c0-4b53-a705-f204de847c9c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:09:58.437302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-861f63f1-b7c0-4b53-a705-f204de847c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:09:58.454095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:58.569247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 475/964] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:09:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:09:58.663392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112c449-3aab-4bce-829d-d7ba44b63c49 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:58.834680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b112c449-3aab-4bce-829d-d7ba44b63c49 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:58.904593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0365684-f807-4655-8ddb-db9a64c5b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:58.905724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0365684-f807-4655-8ddb-db9a64c5b0ce None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:58.906251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0365684-f807-4655-8ddb-db9a64c5b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:58.906738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0365684-f807-4655-8ddb-db9a64c5b0ce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:58.927752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0365684-f807-4655-8ddb-db9a64c5b0ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-f0365684-f807-4655-8ddb-db9a64c5b0ce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:09:58.936843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:58.937862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:58.938767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:58.938767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:58.938767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:58.938767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:58.938767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:09:58.938767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:09:58.938767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:09:58.938767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:09:58.938767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:09:58.938767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:09:58.938767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 16:09:58.938767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:09:58.939441 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 490/965] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Sep 21 16:09:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:58.949135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f63f1-b7c0-4b53-a705-f204de847c9c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:58.977114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0365684-f807-4655-8ddb-db9a64c5b0ce None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:58.997948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:59.148689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f63f1-b7c0-4b53-a705-f204de847c9c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:59.155090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0365684-f807-4655-8ddb-db9a64c5b0ce None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:59.221655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba100b0-c19b-48df-b24d-83d4801188a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:59.222861 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba100b0-c19b-48df-b24d-83d4801188a6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:59.223632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba100b0-c19b-48df-b24d-83d4801188a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:59.224223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba100b0-c19b-48df-b24d-83d4801188a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:59.225557 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba100b0-c19b-48df-b24d-83d4801188a6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:59.226207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba100b0-c19b-48df-b24d-83d4801188a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:59.229132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730efaf5-9928-4fb8-b6f8-f98560bea5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:59.229366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba100b0-c19b-48df-b24d-83d4801188a6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:59.230425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730efaf5-9928-4fb8-b6f8-f98560bea5b8 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:59.231002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730efaf5-9928-4fb8-b6f8-f98560bea5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:59.231561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730efaf5-9928-4fb8-b6f8-f98560bea5b8 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:59.232891 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730efaf5-9928-4fb8-b6f8-f98560bea5b8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:59.235692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730efaf5-9928-4fb8-b6f8-f98560bea5b8 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:59.247434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 476/966] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Sep 21 16:09:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:59.258595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba100b0-c19b-48df-b24d-83d4801188a6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:59.265619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:59.268459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 491/967] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:09:58 2022] POST /identity/v3/services => generated 242 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:09:59.478422 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba100b0-c19b-48df-b24d-83d4801188a6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:59.557680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf509bb-efb8-4019-8c6b-784d5d224c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:59.559219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf509bb-efb8-4019-8c6b-784d5d224c74 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:59.560017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf509bb-efb8-4019-8c6b-784d5d224c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:59.560763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf509bb-efb8-4019-8c6b-784d5d224c74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:59.582039 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf509bb-efb8-4019-8c6b-784d5d224c74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:59.585155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf509bb-efb8-4019-8c6b-784d5d224c74 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-bcf509bb-efb8-4019-8c6b-784d5d224c74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:09:59.615254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:09:59.616434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:09:59.617802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:09:59.617802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:09:59.617802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:09:59.621591 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 477/968] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:09:59.631904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730efaf5-9928-4fb8-b6f8-f98560bea5b8 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:09:59.782438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:09:59.853210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730efaf5-9928-4fb8-b6f8-f98560bea5b8 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:09:59.923112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbff58-9c66-42fc-8831-2653e681f939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:09:59.924060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbff58-9c66-42fc-8831-2653e681f939 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:09:59.924524 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbff58-9c66-42fc-8831-2653e681f939 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:09:59.925008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cbff58-9c66-42fc-8831-2653e681f939 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:09:59.926166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbff58-9c66-42fc-8831-2653e681f939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:09:59.927292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbff58-9c66-42fc-8831-2653e681f939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:09:59.929469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cbff58-9c66-42fc-8831-2653e681f939 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:09:59.945046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 492/969] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:09:59.950958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf509bb-efb8-4019-8c6b-784d5d224c74 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:00.015845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:00.069601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf509bb-efb8-4019-8c6b-784d5d224c74 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:00.106680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847618e-3246-4d8d-b47d-040f91eb0486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:00.107335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847618e-3246-4d8d-b47d-040f91eb0486 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:00.107617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847618e-3246-4d8d-b47d-040f91eb0486 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:00.107892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847618e-3246-4d8d-b47d-040f91eb0486 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:00.117070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847618e-3246-4d8d-b47d-040f91eb0486 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:00.118611 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847618e-3246-4d8d-b47d-040f91eb0486 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-7847618e-3246-4d8d-b47d-040f91eb0486 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:10:00.131310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:00.132395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:00.133371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:00.133371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:00.133371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:10:00.133371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:10:00.133371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:10:00.133371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:10:00.133371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:10:00.133371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:10:00.133371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 21 16:10:00.133371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:00.133371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 478/970] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Sep 21 16:09:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:00.140286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbff58-9c66-42fc-8831-2653e681f939 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:00.275779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cbff58-9c66-42fc-8831-2653e681f939 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:00.341464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:00.347580 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bf5c4b-903f-4fd4-87cc-e9561cc3f80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:00.349443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf5c4b-903f-4fd4-87cc-e9561cc3f80b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:00.350335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf5c4b-903f-4fd4-87cc-e9561cc3f80b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:00.350818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bf5c4b-903f-4fd4-87cc-e9561cc3f80b None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:00.352153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf5c4b-903f-4fd4-87cc-e9561cc3f80b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:00.352694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf5c4b-903f-4fd4-87cc-e9561cc3f80b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:00.355342 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bf5c4b-903f-4fd4-87cc-e9561cc3f80b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:00.389646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 493/971] 10.222.0.64 () {56 vars in 1149 bytes} [Wed Sep 21 16:10:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:00.396074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847618e-3246-4d8d-b47d-040f91eb0486 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:00.399868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c18b44-2559-4e78-ae17-6f551291acdf None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:00.400154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c18b44-2559-4e78-ae17-6f551291acdf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:00.400425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c18b44-2559-4e78-ae17-6f551291acdf None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:00.401887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 494/972] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:10:00.407216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c2a8f-1621-45f9-8d2c-430db965e133 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:00.407496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c2a8f-1621-45f9-8d2c-430db965e133 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:00.407756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c2a8f-1621-45f9-8d2c-430db965e133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:00.426539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f90c2a8f-1621-45f9-8d2c-430db965e133 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:00.427748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f90c2a8f-1621-45f9-8d2c-430db965e133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:00.521334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 495/973] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:10:00.523269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02382a5e-1e4d-4134-8949-e8b35b82cd5b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:00.523684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02382a5e-1e4d-4134-8949-e8b35b82cd5b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:00.524082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02382a5e-1e4d-4134-8949-e8b35b82cd5b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:00.525732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 496/974] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:00.534413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925a812-df25-4792-91d4-271d0a1036fb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:00.534817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925a812-df25-4792-91d4-271d0a1036fb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:00.535201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d925a812-df25-4792-91d4-271d0a1036fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:00.539877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847618e-3246-4d8d-b47d-040f91eb0486 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:00.564338 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d925a812-df25-4792-91d4-271d0a1036fb None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:00.565084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:00.568997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d925a812-df25-4792-91d4-271d0a1036fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:00.603320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f7858-0f68-4b42-9f8f-fd5ad335e4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:00.604623 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f7858-0f68-4b42-9f8f-fd5ad335e4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:00.605266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f7858-0f68-4b42-9f8f-fd5ad335e4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:00.605915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f7858-0f68-4b42-9f8f-fd5ad335e4d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:00.701890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 497/975] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:00.738393 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f7858-0f68-4b42-9f8f-fd5ad335e4d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:00.741498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f7858-0f68-4b42-9f8f-fd5ad335e4d4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:00.927882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 479/976] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:10:00 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:10:00.936193 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925a812-df25-4792-91d4-271d0a1036fb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:00.998535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f7858-0f68-4b42-9f8f-fd5ad335e4d4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:01.015266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:01.026457 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:01.104224 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d925a812-df25-4792-91d4-271d0a1036fb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:01.144137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6697cfc-2dde-4b03-960f-09bcd6500c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:01.145551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6697cfc-2dde-4b03-960f-09bcd6500c6c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:01.145834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6697cfc-2dde-4b03-960f-09bcd6500c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:01.146139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6697cfc-2dde-4b03-960f-09bcd6500c6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:01.154846 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f7858-0f68-4b42-9f8f-fd5ad335e4d4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:01.157454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6697cfc-2dde-4b03-960f-09bcd6500c6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:01.158974 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6697cfc-2dde-4b03-960f-09bcd6500c6c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-c6697cfc-2dde-4b03-960f-09bcd6500c6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:10:01.175612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:01.177296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:10:01.178306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:10:01.178306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:01.178306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:01.178306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 498/977] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:01.178545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6697cfc-2dde-4b03-960f-09bcd6500c6c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:01.199788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193c40-ae99-431d-a0bf-aa8bf073ce9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:01.200429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193c40-ae99-431d-a0bf-aa8bf073ce9d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:01.200702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193c40-ae99-431d-a0bf-aa8bf073ce9d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:01.200972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78193c40-ae99-431d-a0bf-aa8bf073ce9d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:01.201563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78193c40-ae99-431d-a0bf-aa8bf073ce9d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:01.203454 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78193c40-ae99-431d-a0bf-aa8bf073ce9d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-78193c40-ae99-431d-a0bf-aa8bf073ce9d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:01.214518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:01.215571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:01.216408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 480/978] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Sep 21 16:10:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:10:01.217216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193c40-ae99-431d-a0bf-aa8bf073ce9d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:01.315996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6697cfc-2dde-4b03-960f-09bcd6500c6c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:01.318477 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:01.324942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:01.367407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78193c40-ae99-431d-a0bf-aa8bf073ce9d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:01.380634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95874d08-3eb9-450e-a639-69f25c99a972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:01.381669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95874d08-3eb9-450e-a639-69f25c99a972 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:01.381791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95874d08-3eb9-450e-a639-69f25c99a972 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:01.382254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95874d08-3eb9-450e-a639-69f25c99a972 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:01.383366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95874d08-3eb9-450e-a639-69f25c99a972 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:01.385726 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95874d08-3eb9-450e-a639-69f25c99a972 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:01.389883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: WARNING keystone.server.flask.common [None req-95874d08-3eb9-450e-a639-69f25c99a972 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 21 16:10:01.409246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.sql.core [None req-95874d08-3eb9-450e-a639-69f25c99a972 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 21 16:10:01.409246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 21 16:10:01.409246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [parameters: {'user_id': '03199228cc9248d48599a52b7a7d2d1f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 21 16:10:01.409246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81426) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 21 16:10:01.429880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aa0a6-8a9b-407c-8d08-ea50366d64cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:01.430786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aa0a6-8a9b-407c-8d08-ea50366d64cd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:01.431288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aa0a6-8a9b-407c-8d08-ea50366d64cd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:01.431745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aa0a6-8a9b-407c-8d08-ea50366d64cd None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:01.432829 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888aa0a6-8a9b-407c-8d08-ea50366d64cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:01.433342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888aa0a6-8a9b-407c-8d08-ea50366d64cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:01.435770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888aa0a6-8a9b-407c-8d08-ea50366d64cd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-95874d08-3eb9-450e-a639-69f25c99a972 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application conn.query(q) Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result.read() Sep 21 16:10:01.438392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 16:10:01.439261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application next(self.gen) Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application yield resource Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application next(self.gen) Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.session.rollback() Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.force_reraise() Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise self.value Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application session.commit() Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 21 16:10:01.440206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._prepare_impl() Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.session.flush() Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._flush(objects) Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application compat.raise_( Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application flush_context.execute() Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rec.execute(self) Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 21 16:10:01.441140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 21 16:10:01.443941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result = self._query(query) Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application conn.query(q) Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application result.read() Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 21 16:10:01.447026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [parameters: {'user_id': '03199228cc9248d48599a52b7a7d2d1f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:01.450220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 21 16:10:01.452230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 21 16:10:01.454759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 21 16:10:01.454759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 21 16:10:01.454759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 21 16:10:01.454759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 21 16:10:01.454759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 21 16:10:01.454759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:01.454759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 499/979] 10.222.0.64 () {60 vars in 1131 bytes} [Wed Sep 21 16:10:01 2022] POST /identity/v3/users => generated 160 bytes in 262 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:01.454759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95874d08-3eb9-450e-a639-69f25c99a972 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:01.465787 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 481/980] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:01.474709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aa0a6-8a9b-407c-8d08-ea50366d64cd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:01.611393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95874d08-3eb9-450e-a639-69f25c99a972 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:01.628204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aa0a6-8a9b-407c-8d08-ea50366d64cd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:01.653928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5b1f3-88e5-41b5-a7bc-6e0ce688b2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:01.654607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5b1f3-88e5-41b5-a7bc-6e0ce688b2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:01.654880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5b1f3-88e5-41b5-a7bc-6e0ce688b2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:01.655157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5b1f3-88e5-41b5-a7bc-6e0ce688b2d1 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:01.655807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5b1f3-88e5-41b5-a7bc-6e0ce688b2d1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:01.656113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5b1f3-88e5-41b5-a7bc-6e0ce688b2d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:01.657773 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5b1f3-88e5-41b5-a7bc-6e0ce688b2d1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:01.670099 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 500/981] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Sep 21 16:10:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:01.670099 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5b1f3-88e5-41b5-a7bc-6e0ce688b2d1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:01.705754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36459d78-5ed8-4637-8e74-daae6dc254ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:01.706772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36459d78-5ed8-4637-8e74-daae6dc254ca None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:01.707266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36459d78-5ed8-4637-8e74-daae6dc254ca None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:01.707912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36459d78-5ed8-4637-8e74-daae6dc254ca None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:01.708019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:01.713886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:01.724037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36459d78-5ed8-4637-8e74-daae6dc254ca None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:01.727161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36459d78-5ed8-4637-8e74-daae6dc254ca None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-36459d78-5ed8-4637-8e74-daae6dc254ca None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:10:01.752573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:01.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:01.765574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:01.765574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:01.765574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:10:01.765574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:10:01.765574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:10:01.765574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:10:01.765574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:10:01.765574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:10:01.765574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 21 16:10:01.765574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:01.765574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 482/982] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:10:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:01.765574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36459d78-5ed8-4637-8e74-daae6dc254ca None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:01.836267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5b1f3-88e5-41b5-a7bc-6e0ce688b2d1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:01.874331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36459d78-5ed8-4637-8e74-daae6dc254ca None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:01.887493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf00f52-04e6-49ec-bdc3-ce9c817c495e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:01.888259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf00f52-04e6-49ec-bdc3-ce9c817c495e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:01.888604 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf00f52-04e6-49ec-bdc3-ce9c817c495e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:01.888951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf00f52-04e6-49ec-bdc3-ce9c817c495e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:01.897271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf00f52-04e6-49ec-bdc3-ce9c817c495e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:01.898998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf00f52-04e6-49ec-bdc3-ce9c817c495e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-bbf00f52-04e6-49ec-bdc3-ce9c817c495e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:10:01.912116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:01.922284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:01.924103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:10:01.924103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:10:01.924103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:10:01.924103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:10:01.924103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:10:01.924103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:10:01.924103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:10:01.924103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:01.924103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 501/983] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:10:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:01.924103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf00f52-04e6-49ec-bdc3-ce9c817c495e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:01.925165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d923caed-e662-40ee-a656-bc526c7ebed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:01.925165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923caed-e662-40ee-a656-bc526c7ebed4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:01.925165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923caed-e662-40ee-a656-bc526c7ebed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:01.925165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d923caed-e662-40ee-a656-bc526c7ebed4 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:01.925652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923caed-e662-40ee-a656-bc526c7ebed4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:01.925652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923caed-e662-40ee-a656-bc526c7ebed4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:01.925652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d923caed-e662-40ee-a656-bc526c7ebed4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:01.930913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 483/984] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Sep 21 16:10:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:01.932422 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01324d94-90e6-4456-ad39-a3df409b1abd None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:01.932832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01324d94-90e6-4456-ad39-a3df409b1abd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:01.933681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01324d94-90e6-4456-ad39-a3df409b1abd None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:01.935181 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 484/985] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:01.936007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01324d94-90e6-4456-ad39-a3df409b1abd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:02.020937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:02.034860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:02.049819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf00f52-04e6-49ec-bdc3-ce9c817c495e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:02.073212 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01324d94-90e6-4456-ad39-a3df409b1abd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:02.091930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd0035b-a595-41cb-86c1-51f5ab5b21ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:02.092827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0035b-a595-41cb-86c1-51f5ab5b21ae None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:02.093154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0035b-a595-41cb-86c1-51f5ab5b21ae None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:02.093462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0035b-a595-41cb-86c1-51f5ab5b21ae None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:02.094268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0035b-a595-41cb-86c1-51f5ab5b21ae None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:02.094644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0035b-a595-41cb-86c1-51f5ab5b21ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:02.096468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd0035b-a595-41cb-86c1-51f5ab5b21ae None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:02.110997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb3d47-a16f-44b9-acfd-5a53a8495308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:02.111619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb3d47-a16f-44b9-acfd-5a53a8495308 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:02.111899 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb3d47-a16f-44b9-acfd-5a53a8495308 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:02.112174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb3d47-a16f-44b9-acfd-5a53a8495308 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:02.112873 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb3d47-a16f-44b9-acfd-5a53a8495308 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:02.113179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb3d47-a16f-44b9-acfd-5a53a8495308 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:02.114708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb3d47-a16f-44b9-acfd-5a53a8495308 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:02.128212 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 485/986] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:10:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:02.129125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb3d47-a16f-44b9-acfd-5a53a8495308 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:02.138173 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 502/987] 10.222.0.64 () {56 vars in 1101 bytes} [Wed Sep 21 16:10:01 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:02.139676 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4b94-ec2e-4e3e-918d-59558c7f40df None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:02.139963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4b94-ec2e-4e3e-918d-59558c7f40df None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:02.140235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc4b94-ec2e-4e3e-918d-59558c7f40df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:02.171958 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-37fc4b94-ec2e-4e3e-918d-59558c7f40df None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:02.174810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-37fc4b94-ec2e-4e3e-918d-59558c7f40df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:02.306307 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 503/988] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:02.306307 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc4b94-ec2e-4e3e-918d-59558c7f40df None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:02.335874 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb3d47-a16f-44b9-acfd-5a53a8495308 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:02.370506 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:02.389686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a362e35-6e4b-4664-bbc4-83d91ee26d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:02.390343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a362e35-6e4b-4664-bbc4-83d91ee26d6f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:02.390622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a362e35-6e4b-4664-bbc4-83d91ee26d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:02.390890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a362e35-6e4b-4664-bbc4-83d91ee26d6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:02.391646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a362e35-6e4b-4664-bbc4-83d91ee26d6f None admin] RBAC: Adding query filter params (interface=admin, service_id=cd1e5d1c3cbf42a78892a179dad103bb, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:02.391955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a362e35-6e4b-4664-bbc4-83d91ee26d6f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:02.393927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a362e35-6e4b-4664-bbc4-83d91ee26d6f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:02.408280 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 486/989] 10.222.0.64 () {56 vars in 1249 bytes} [Wed Sep 21 16:10:02 2022] GET /identity/v3/endpoints?service_id=cd1e5d1c3cbf42a78892a179dad103bb&interface=admin®ion_id=RegionOne => generated 193 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:02.409968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a362e35-6e4b-4664-bbc4-83d91ee26d6f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:02.442711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:02.467969 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc4b94-ec2e-4e3e-918d-59558c7f40df None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:02.545324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f046d-3b3a-461a-8b67-0874c267afa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:02.546537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f046d-3b3a-461a-8b67-0874c267afa3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:02.547113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f046d-3b3a-461a-8b67-0874c267afa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:02.547700 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f046d-3b3a-461a-8b67-0874c267afa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:02.549091 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f046d-3b3a-461a-8b67-0874c267afa3 None admin] RBAC: Adding query filter params (role.id=8e59f99483d148c09246abf67508a11b, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=8da8d12ac74644f280ba31da1715f603) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:02.549748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f046d-3b3a-461a-8b67-0874c267afa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:02.553041 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f046d-3b3a-461a-8b67-0874c267afa3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:02.564140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 504/990] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:10:02 2022] GET /identity/v3/role_assignments?user.id=8da8d12ac74644f280ba31da1715f603&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=8e59f99483d148c09246abf67508a11b => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:02.593091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a362e35-6e4b-4664-bbc4-83d91ee26d6f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:02.655726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651a1c9-80dd-4379-ac19-1b9688d1aa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:02.656675 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651a1c9-80dd-4379-ac19-1b9688d1aa5f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:02.657094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651a1c9-80dd-4379-ac19-1b9688d1aa5f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:02.657506 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651a1c9-80dd-4379-ac19-1b9688d1aa5f None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:02.658501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651a1c9-80dd-4379-ac19-1b9688d1aa5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:02.661326 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651a1c9-80dd-4379-ac19-1b9688d1aa5f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:02.678354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 487/991] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Sep 21 16:10:02 2022] GET /identity/v3/services => generated 1727 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:10:02.699918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f046d-3b3a-461a-8b67-0874c267afa3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:02.878187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f046d-3b3a-461a-8b67-0874c267afa3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:02.938614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde09239-ff78-4a57-94d7-5954986ca725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:02.939724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde09239-ff78-4a57-94d7-5954986ca725 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:02.940209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde09239-ff78-4a57-94d7-5954986ca725 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:02.940723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde09239-ff78-4a57-94d7-5954986ca725 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:02.942394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde09239-ff78-4a57-94d7-5954986ca725 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:02.944870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde09239-ff78-4a57-94d7-5954986ca725 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-bde09239-ff78-4a57-94d7-5954986ca725 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:02.962687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:02.963733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 16:10:02.964596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:02.967351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 505/992] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:10:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:02.972048 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651a1c9-80dd-4379-ac19-1b9688d1aa5f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:03.062037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:03.153907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651a1c9-80dd-4379-ac19-1b9688d1aa5f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:03.209394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3968eb-3f4d-4a6d-a17f-c632646e88d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:03.210010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3968eb-3f4d-4a6d-a17f-c632646e88d0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:03.210364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3968eb-3f4d-4a6d-a17f-c632646e88d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:03.210559 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3968eb-3f4d-4a6d-a17f-c632646e88d0 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:03.211204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3968eb-3f4d-4a6d-a17f-c632646e88d0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:03.211498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3968eb-3f4d-4a6d-a17f-c632646e88d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:03.212866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3968eb-3f4d-4a6d-a17f-c632646e88d0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:03.222450 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 488/993] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:10:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:03.232373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde09239-ff78-4a57-94d7-5954986ca725 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:03.240039 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:03.444819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde09239-ff78-4a57-94d7-5954986ca725 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:03.523218 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36490e4b-f631-443a-888b-ba20939cf41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:03.524223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36490e4b-f631-443a-888b-ba20939cf41e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:03.524669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36490e4b-f631-443a-888b-ba20939cf41e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:03.525133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36490e4b-f631-443a-888b-ba20939cf41e None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:03.526321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36490e4b-f631-443a-888b-ba20939cf41e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:03.526915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36490e4b-f631-443a-888b-ba20939cf41e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:03.529345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36490e4b-f631-443a-888b-ba20939cf41e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:03.539617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 506/994] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:10:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:03.548757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3968eb-3f4d-4a6d-a17f-c632646e88d0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:03.733511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3968eb-3f4d-4a6d-a17f-c632646e88d0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:03.757006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:03.796051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917997b1-fbb5-462b-81c9-89075123f60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:03.797576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917997b1-fbb5-462b-81c9-89075123f60d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:03.798252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917997b1-fbb5-462b-81c9-89075123f60d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:03.798877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917997b1-fbb5-462b-81c9-89075123f60d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:03.800346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917997b1-fbb5-462b-81c9-89075123f60d None admin] RBAC: Adding query filter params (interface=public, service_id=dc447bc2ef7a492ba374702a70675dda, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:03.801105 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917997b1-fbb5-462b-81c9-89075123f60d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:03.804026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917997b1-fbb5-462b-81c9-89075123f60d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:03.813936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 489/995] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Sep 21 16:10:03 2022] GET /identity/v3/endpoints?service_id=dc447bc2ef7a492ba374702a70675dda&interface=public®ion_id=RegionOne => generated 194 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:03.819920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36490e4b-f631-443a-888b-ba20939cf41e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:03.921528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:03.973585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36490e4b-f631-443a-888b-ba20939cf41e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:04.018478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b922d-b06f-4b6e-97ed-63738e6f7d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:04.018989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b922d-b06f-4b6e-97ed-63738e6f7d52 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:04.019306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b922d-b06f-4b6e-97ed-63738e6f7d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:04.019584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b922d-b06f-4b6e-97ed-63738e6f7d52 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:04.020239 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b922d-b06f-4b6e-97ed-63738e6f7d52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:04.021600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b922d-b06f-4b6e-97ed-63738e6f7d52 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:04.029097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 507/996] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Sep 21 16:10:03 2022] GET /identity/v3/services => generated 1727 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:10:04.949442 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980568e-b51c-4c83-96b6-7ddfbb299784 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:04.950291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980568e-b51c-4c83-96b6-7ddfbb299784 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:04.950291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980568e-b51c-4c83-96b6-7ddfbb299784 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:04.951625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 490/997] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:10:04.958902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff7445-b4a6-47f7-a387-24cca95e782f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:04.959416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff7445-b4a6-47f7-a387-24cca95e782f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:04.960033 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaff7445-b4a6-47f7-a387-24cca95e782f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:04.962086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2935836-13e5-4abe-9750-d1cb19f469a4 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:04.962371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2935836-13e5-4abe-9750-d1cb19f469a4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:04.962644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2935836-13e5-4abe-9750-d1cb19f469a4 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:04.964110 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 491/998] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:10:04.969126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9d3ca-8812-45f4-a268-ec9bef6d67da None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:04.969411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9d3ca-8812-45f4-a268-ec9bef6d67da None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:04.969687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9d3ca-8812-45f4-a268-ec9bef6d67da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:04.988979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f6f9d3ca-8812-45f4-a268-ec9bef6d67da None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:04.990411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f6f9d3ca-8812-45f4-a268-ec9bef6d67da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:04.993166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-aaff7445-b4a6-47f7-a387-24cca95e782f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:04.995326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-aaff7445-b4a6-47f7-a387-24cca95e782f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:05.090079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 492/999] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:10:05.090302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 508/1000] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:10:05.355803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9d3ca-8812-45f4-a268-ec9bef6d67da None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:05.376521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff7445-b4a6-47f7-a387-24cca95e782f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:05.450475 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:05.478798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9d3ca-8812-45f4-a268-ec9bef6d67da None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:05.492832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:05.506156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaff7445-b4a6-47f7-a387-24cca95e782f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:05.529575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751aa4dc-e69b-41d2-99ec-64ec292098dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:05.530729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aa4dc-e69b-41d2-99ec-64ec292098dc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:05.531470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aa4dc-e69b-41d2-99ec-64ec292098dc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:05.532001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751aa4dc-e69b-41d2-99ec-64ec292098dc None admin] PATH_INFO: `/v3/services/image` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:05.533500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751aa4dc-e69b-41d2-99ec-64ec292098dc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:05.536029 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751aa4dc-e69b-41d2-99ec-64ec292098dc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-751aa4dc-e69b-41d2-99ec-64ec292098dc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:05.551926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:05.553024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:05.553867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 493/1001] 10.222.0.64 () {56 vars in 1105 bytes} [Wed Sep 21 16:10:05 2022] GET /identity/v3/services/image => generated 86 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:10:05.559537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751aa4dc-e69b-41d2-99ec-64ec292098dc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:05.569673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665c0cb-75f9-43e7-9cf1-d29373a90543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:05.570588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665c0cb-75f9-43e7-9cf1-d29373a90543 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:05.571014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665c0cb-75f9-43e7-9cf1-d29373a90543 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:05.572214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665c0cb-75f9-43e7-9cf1-d29373a90543 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:05.572957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665c0cb-75f9-43e7-9cf1-d29373a90543 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:05.574984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665c0cb-75f9-43e7-9cf1-d29373a90543 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-0665c0cb-75f9-43e7-9cf1-d29373a90543 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:05.588538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:05.589564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:05.590447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 509/1002] 10.222.0.64 () {56 vars in 1126 bytes} [Wed Sep 21 16:10:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:10:05.599850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665c0cb-75f9-43e7-9cf1-d29373a90543 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:05.712946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751aa4dc-e69b-41d2-99ec-64ec292098dc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:05.763946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665c0cb-75f9-43e7-9cf1-d29373a90543 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:05.773498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed293b9-80dc-4a55-9cc9-bf25a3a3ac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:05.774404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed293b9-80dc-4a55-9cc9-bf25a3a3ac35 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:05.774838 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed293b9-80dc-4a55-9cc9-bf25a3a3ac35 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:05.775294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed293b9-80dc-4a55-9cc9-bf25a3a3ac35 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:05.776345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed293b9-80dc-4a55-9cc9-bf25a3a3ac35 None admin] RBAC: Adding query filter params (name=image) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:05.776825 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed293b9-80dc-4a55-9cc9-bf25a3a3ac35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:05.778693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed293b9-80dc-4a55-9cc9-bf25a3a3ac35 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:05.786010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 494/1003] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:05.794597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed293b9-80dc-4a55-9cc9-bf25a3a3ac35 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:05.805742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:05.819693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e9ba8-e2aa-42fd-99c9-c4174039925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:05.820680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e9ba8-e2aa-42fd-99c9-c4174039925c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:05.821089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e9ba8-e2aa-42fd-99c9-c4174039925c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:05.821734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e9ba8-e2aa-42fd-99c9-c4174039925c None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:05.822724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e9ba8-e2aa-42fd-99c9-c4174039925c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:05.823136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e9ba8-e2aa-42fd-99c9-c4174039925c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:05.824941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e9ba8-e2aa-42fd-99c9-c4174039925c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:05.831310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 510/1004] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Sep 21 16:10:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:05.837662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e9ba8-e2aa-42fd-99c9-c4174039925c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:05.840871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:05.942177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed293b9-80dc-4a55-9cc9-bf25a3a3ac35 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:05.985372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f4f24-2f26-4e1b-bdb3-ae7f0b524dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:05.986425 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f4f24-2f26-4e1b-bdb3-ae7f0b524dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:05.986425 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f4f24-2f26-4e1b-bdb3-ae7f0b524dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:05.986425 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f4f24-2f26-4e1b-bdb3-ae7f0b524dc8 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:05.987119 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f4f24-2f26-4e1b-bdb3-ae7f0b524dc8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:05.987411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f4f24-2f26-4e1b-bdb3-ae7f0b524dc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:05.988831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f4f24-2f26-4e1b-bdb3-ae7f0b524dc8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:05.999457 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 495/1005] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:05 2022] GET /identity/v3/services?type=image => generated 122 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:06.008922 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e9ba8-e2aa-42fd-99c9-c4174039925c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:06.049505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2866690-7d25-4dc9-a821-54381425dba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:06.050163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2866690-7d25-4dc9-a821-54381425dba0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:06.050442 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2866690-7d25-4dc9-a821-54381425dba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:06.050736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2866690-7d25-4dc9-a821-54381425dba0 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:06.051403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2866690-7d25-4dc9-a821-54381425dba0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:06.051705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2866690-7d25-4dc9-a821-54381425dba0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:06.053114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2866690-7d25-4dc9-a821-54381425dba0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:06.063715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 511/1006] 10.222.0.64 () {56 vars in 1122 bytes} [Wed Sep 21 16:10:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:06.065414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f4f24-2f26-4e1b-bdb3-ae7f0b524dc8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:06.175243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:06.179876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f4f24-2f26-4e1b-bdb3-ae7f0b524dc8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:06.218475 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe0541-8a2d-4f6a-81ab-28d6dec0a0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:06.219103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe0541-8a2d-4f6a-81ab-28d6dec0a0e0 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:06.219379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe0541-8a2d-4f6a-81ab-28d6dec0a0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:06.219652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe0541-8a2d-4f6a-81ab-28d6dec0a0e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:06.220334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe0541-8a2d-4f6a-81ab-28d6dec0a0e0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:06.221719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fe0541-8a2d-4f6a-81ab-28d6dec0a0e0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:06.246195 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 496/1007] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:10:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:10:06.444258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0da818-5d5c-4194-a987-6e71efadd3f1 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:06.444545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0da818-5d5c-4194-a987-6e71efadd3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:06.444822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0da818-5d5c-4194-a987-6e71efadd3f1 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:06.447423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 512/1008] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:06.452112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78e51f-b7f3-4a92-8ede-1784457207f6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:06.452419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78e51f-b7f3-4a92-8ede-1784457207f6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:06.452678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78e51f-b7f3-4a92-8ede-1784457207f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:06.472037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bb78e51f-b7f3-4a92-8ede-1784457207f6 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:06.473791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bb78e51f-b7f3-4a92-8ede-1784457207f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:06.572658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 497/1009] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:06 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:06.903607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0da818-5d5c-4194-a987-6e71efadd3f1 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:06.967017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:07.021606 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0da818-5d5c-4194-a987-6e71efadd3f1 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:07.067901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c7d8a-60e6-49f4-9620-94b568e446c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:07.068549 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c7d8a-60e6-49f4-9620-94b568e446c4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:07.068826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c7d8a-60e6-49f4-9620-94b568e446c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:07.069094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c7d8a-60e6-49f4-9620-94b568e446c4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:07.069686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c7d8a-60e6-49f4-9620-94b568e446c4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:07.071113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c7d8a-60e6-49f4-9620-94b568e446c4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-fd2c7d8a-60e6-49f4-9620-94b568e446c4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:07.083277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:07.085096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 21 16:10:07.087568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:07.089631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 513/1010] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:10:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:10:07.090478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78e51f-b7f3-4a92-8ede-1784457207f6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:07.185449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:07.207845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78e51f-b7f3-4a92-8ede-1784457207f6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:07.249102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3e989-921d-4c93-a703-03a480a242d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:07.249886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3e989-921d-4c93-a703-03a480a242d2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:07.250338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3e989-921d-4c93-a703-03a480a242d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:07.250686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3e989-921d-4c93-a703-03a480a242d2 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:07.251740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3e989-921d-4c93-a703-03a480a242d2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:07.252229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3e989-921d-4c93-a703-03a480a242d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:07.254272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3e989-921d-4c93-a703-03a480a242d2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:07.265502 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 498/1011] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:10:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:07.267905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c7d8a-60e6-49f4-9620-94b568e446c4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:07.397193 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c7d8a-60e6-49f4-9620-94b568e446c4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:07.419756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:07.445189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6944f903-c7eb-4919-a8ac-80f3f685dcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:07.445742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944f903-c7eb-4919-a8ac-80f3f685dcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:07.446025 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944f903-c7eb-4919-a8ac-80f3f685dcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:07.446601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944f903-c7eb-4919-a8ac-80f3f685dcc1 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:07.447277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944f903-c7eb-4919-a8ac-80f3f685dcc1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:07.447585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944f903-c7eb-4919-a8ac-80f3f685dcc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:07.449017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6944f903-c7eb-4919-a8ac-80f3f685dcc1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:07.459912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 514/1012] 10.222.0.64 () {56 vars in 1112 bytes} [Wed Sep 21 16:10:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:07.463189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3e989-921d-4c93-a703-03a480a242d2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:07.637968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3e989-921d-4c93-a703-03a480a242d2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:07.698892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4401e-4414-4301-a7b2-3a31909d3a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:07.699764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4401e-4414-4301-a7b2-3a31909d3a86 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:07.700276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4401e-4414-4301-a7b2-3a31909d3a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:07.700770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4401e-4414-4301-a7b2-3a31909d3a86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:07.701846 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4401e-4414-4301-a7b2-3a31909d3a86 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:07.704446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4401e-4414-4301-a7b2-3a31909d3a86 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:07.736664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 499/1013] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:10:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:10:08.553311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffddf22-1826-4bbc-bda1-4c73507c8c26 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:08.553866 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffddf22-1826-4bbc-bda1-4c73507c8c26 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:08.553866 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffddf22-1826-4bbc-bda1-4c73507c8c26 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:08.555445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 515/1014] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:08.561297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77970b90-5b41-4bf5-a9be-fe8d2bcf3a12 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:08.561819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77970b90-5b41-4bf5-a9be-fe8d2bcf3a12 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:08.561819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77970b90-5b41-4bf5-a9be-fe8d2bcf3a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:08.569922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:08.584643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-77970b90-5b41-4bf5-a9be-fe8d2bcf3a12 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:08.585859 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-77970b90-5b41-4bf5-a9be-fe8d2bcf3a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:08.650411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14825c02-1938-49eb-b769-fed2a688c81f None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:08.650846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14825c02-1938-49eb-b769-fed2a688c81f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:08.651384 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14825c02-1938-49eb-b769-fed2a688c81f None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:08.653651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 516/1015] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:08.662121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5eb4d9-c5f8-469c-a835-2d712adab3e0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:08.662620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5eb4d9-c5f8-469c-a835-2d712adab3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:08.663083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5eb4d9-c5f8-469c-a835-2d712adab3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:08.688159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 500/1016] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:08 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:08.688681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ea5eb4d9-c5f8-469c-a835-2d712adab3e0 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:08.690158 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ea5eb4d9-c5f8-469c-a835-2d712adab3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:08.772040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 517/1017] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:08 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:08.966112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77970b90-5b41-4bf5-a9be-fe8d2bcf3a12 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:09.080735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77970b90-5b41-4bf5-a9be-fe8d2bcf3a12 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:09.115820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:09.123168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e45dbe-4364-4a14-a1a2-b8b8de734933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:09.123808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e45dbe-4364-4a14-a1a2-b8b8de734933 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:09.124077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e45dbe-4364-4a14-a1a2-b8b8de734933 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:09.124408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e45dbe-4364-4a14-a1a2-b8b8de734933 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:09.125617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e45dbe-4364-4a14-a1a2-b8b8de734933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:09.127077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e45dbe-4364-4a14-a1a2-b8b8de734933 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:09.130078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 501/1018] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:09.137017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5eb4d9-c5f8-469c-a835-2d712adab3e0 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:09.140781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e45dbe-4364-4a14-a1a2-b8b8de734933 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:09.249366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5eb4d9-c5f8-469c-a835-2d712adab3e0 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:09.264497 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:09.290593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71d94be-e0ec-45a8-b343-eb157b7276be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:09.291207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d94be-e0ec-45a8-b343-eb157b7276be None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:09.291485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d94be-e0ec-45a8-b343-eb157b7276be None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:09.291756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71d94be-e0ec-45a8-b343-eb157b7276be None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:09.292444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d94be-e0ec-45a8-b343-eb157b7276be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:09.293894 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71d94be-e0ec-45a8-b343-eb157b7276be None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:09.319075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e45dbe-4364-4a14-a1a2-b8b8de734933 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:09.330103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 518/1019] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:10:09 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:10:09.368955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c14c10-f635-4418-8d8c-cef1479ffb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:09.369500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c14c10-f635-4418-8d8c-cef1479ffb1a None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:09.369833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c14c10-f635-4418-8d8c-cef1479ffb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:09.370305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c14c10-f635-4418-8d8c-cef1479ffb1a None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:09.371535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c14c10-f635-4418-8d8c-cef1479ffb1a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:09.373020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c14c10-f635-4418-8d8c-cef1479ffb1a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:09.392320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 502/1020] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:10:09 2022] POST /identity/v3/services => generated 242 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:10:10.257612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ee3fa-7993-40be-ad9d-1e623072ce58 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:10.258304 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ee3fa-7993-40be-ad9d-1e623072ce58 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:10.258304 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ee3fa-7993-40be-ad9d-1e623072ce58 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:10.259548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 519/1021] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:10:10.265350 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43598aba-2a95-49d2-a767-b76f13e4dae7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:10.265834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43598aba-2a95-49d2-a767-b76f13e4dae7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:10.265924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43598aba-2a95-49d2-a767-b76f13e4dae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:10.284610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-43598aba-2a95-49d2-a767-b76f13e4dae7 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:10.286400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-43598aba-2a95-49d2-a767-b76f13e4dae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:10.352240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:10.391948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 503/1022] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:10.796621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ee3fa-7993-40be-ad9d-1e623072ce58 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:11.005304 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ee3fa-7993-40be-ad9d-1e623072ce58 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:11.062160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857cb1b-090f-4108-adfb-b3fdde31e74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:11.063224 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857cb1b-090f-4108-adfb-b3fdde31e74d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:11.063722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857cb1b-090f-4108-adfb-b3fdde31e74d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:11.064197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857cb1b-090f-4108-adfb-b3fdde31e74d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:11.086773 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857cb1b-090f-4108-adfb-b3fdde31e74d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a857cb1b-090f-4108-adfb-b3fdde31e74d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:10:11.096719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:11.098002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:11.103345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 520/1023] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Sep 21 16:10:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:11.106749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43598aba-2a95-49d2-a767-b76f13e4dae7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:11.267272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43598aba-2a95-49d2-a767-b76f13e4dae7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:11.326549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e44dd-6305-4ad4-b65d-457ff556a2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:11.327490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e44dd-6305-4ad4-b65d-457ff556a2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:11.327965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e44dd-6305-4ad4-b65d-457ff556a2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:11.328430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e44dd-6305-4ad4-b65d-457ff556a2e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:11.329633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e44dd-6305-4ad4-b65d-457ff556a2e5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:11.330215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e44dd-6305-4ad4-b65d-457ff556a2e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:11.333509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e44dd-6305-4ad4-b65d-457ff556a2e5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:11.353817 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 504/1024] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Sep 21 16:10:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:11.355435 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857cb1b-090f-4108-adfb-b3fdde31e74d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:11.387343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:11.491844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857cb1b-090f-4108-adfb-b3fdde31e74d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:11.544963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe35fa-8ee8-4ca7-8681-8ae7d69bf52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:11.546592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe35fa-8ee8-4ca7-8681-8ae7d69bf52a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:11.547481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe35fa-8ee8-4ca7-8681-8ae7d69bf52a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:11.548082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe35fa-8ee8-4ca7-8681-8ae7d69bf52a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:11.570269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe35fa-8ee8-4ca7-8681-8ae7d69bf52a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:11.573293 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe35fa-8ee8-4ca7-8681-8ae7d69bf52a None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-d0fe35fa-8ee8-4ca7-8681-8ae7d69bf52a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:10:11.606983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:11.608431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:10:11.610285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:10:11.610285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:11.610285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:11.613655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 521/1025] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:11.621288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e44dd-6305-4ad4-b65d-457ff556a2e5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:11.672632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bde2a-8624-46d6-a5a7-ea88dd1a36e9 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:11.672924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bde2a-8624-46d6-a5a7-ea88dd1a36e9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:11.673236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bde2a-8624-46d6-a5a7-ea88dd1a36e9 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:11.674570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 522/1026] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:10:11.681105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154dafb-e85e-4c03-aed8-c607d7f87dfb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:11.681404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154dafb-e85e-4c03-aed8-c607d7f87dfb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:11.681652 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8154dafb-e85e-4c03-aed8-c607d7f87dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:11.685750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:11.707761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8154dafb-e85e-4c03-aed8-c607d7f87dfb None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:11.709121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8154dafb-e85e-4c03-aed8-c607d7f87dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:11.753873 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e44dd-6305-4ad4-b65d-457ff556a2e5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:11.784343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 523/1027] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:10:11.786216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb63db-9b36-4817-8eb9-e1f83cc24e7e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:11.786511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb63db-9b36-4817-8eb9-e1f83cc24e7e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:11.786777 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb63db-9b36-4817-8eb9-e1f83cc24e7e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:11.790292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 524/1028] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:11.797340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c99fa-0c8d-4e2b-818f-54143112414e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:11.797961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c99fa-0c8d-4e2b-818f-54143112414e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:11.798269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c99fa-0c8d-4e2b-818f-54143112414e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:11.798541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c99fa-0c8d-4e2b-818f-54143112414e None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:11.798802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54648362-2c7c-4551-a0fe-fb6b422a32e8 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:11.799179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c99fa-0c8d-4e2b-818f-54143112414e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:11.799344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54648362-2c7c-4551-a0fe-fb6b422a32e8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:11.799494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c99fa-0c8d-4e2b-818f-54143112414e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:11.799631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54648362-2c7c-4551-a0fe-fb6b422a32e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:11.801439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c99fa-0c8d-4e2b-818f-54143112414e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:11.814750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:11.818502 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 505/1029] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:11.824213 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c99fa-0c8d-4e2b-818f-54143112414e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:11.825012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-54648362-2c7c-4551-a0fe-fb6b422a32e8 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:11.826524 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-54648362-2c7c-4551-a0fe-fb6b422a32e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:11.909068 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 525/1030] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:11.940772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c99fa-0c8d-4e2b-818f-54143112414e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:11.949119 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:11.978814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97754fe0-b72b-41ae-aabe-42e3a3e1df50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:11.979436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97754fe0-b72b-41ae-aabe-42e3a3e1df50 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:11.979715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97754fe0-b72b-41ae-aabe-42e3a3e1df50 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:11.979985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97754fe0-b72b-41ae-aabe-42e3a3e1df50 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:11.988278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97754fe0-b72b-41ae-aabe-42e3a3e1df50 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:11.990040 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97754fe0-b72b-41ae-aabe-42e3a3e1df50 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-97754fe0-b72b-41ae-aabe-42e3a3e1df50 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:10:12.003172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:12.004277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:12.010772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:12.010772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:12.010772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:10:12.010772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:10:12.010772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:10:12.010772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:10:12.010772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:10:12.010772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:10:12.010772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:10:12.010772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:12.010772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 506/1031] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Sep 21 16:10:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:12.011393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54648362-2c7c-4551-a0fe-fb6b422a32e8 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:12.152714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54648362-2c7c-4551-a0fe-fb6b422a32e8 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:12.181003 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97754fe0-b72b-41ae-aabe-42e3a3e1df50 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:12.202277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea47ccf-8b7e-446e-8136-a0832e0df2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:12.202892 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea47ccf-8b7e-446e-8136-a0832e0df2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:12.203170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea47ccf-8b7e-446e-8136-a0832e0df2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:12.203433 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea47ccf-8b7e-446e-8136-a0832e0df2b2 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:12.204238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea47ccf-8b7e-446e-8136-a0832e0df2b2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:12.204543 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea47ccf-8b7e-446e-8136-a0832e0df2b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:12.205950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea47ccf-8b7e-446e-8136-a0832e0df2b2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:12.209283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:12.226380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 526/1032] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Sep 21 16:10:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:12.227232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea47ccf-8b7e-446e-8136-a0832e0df2b2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:12.312226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97754fe0-b72b-41ae-aabe-42e3a3e1df50 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:12.341909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea47ccf-8b7e-446e-8136-a0832e0df2b2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:12.361831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa70edb-befc-45c7-9b46-4a257a647921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:12.362930 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa70edb-befc-45c7-9b46-4a257a647921 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:12.363426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa70edb-befc-45c7-9b46-4a257a647921 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:12.363925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa70edb-befc-45c7-9b46-4a257a647921 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:12.365693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa70edb-befc-45c7-9b46-4a257a647921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:12.368480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa70edb-befc-45c7-9b46-4a257a647921 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:12.371637 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 507/1033] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:12.372830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa70edb-befc-45c7-9b46-4a257a647921 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:12.382230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4b2a0-be0d-4db4-9a48-0ce5f0e526d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:12.382883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4b2a0-be0d-4db4-9a48-0ce5f0e526d0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:12.383165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4b2a0-be0d-4db4-9a48-0ce5f0e526d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:12.383439 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4b2a0-be0d-4db4-9a48-0ce5f0e526d0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:12.384065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4b2a0-be0d-4db4-9a48-0ce5f0e526d0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:12.385498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4b2a0-be0d-4db4-9a48-0ce5f0e526d0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-b9a4b2a0-be0d-4db4-9a48-0ce5f0e526d0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:12.395483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:12.398632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:12.400279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 527/1034] 10.222.0.64 () {56 vars in 1105 bytes} [Wed Sep 21 16:10:12 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:12.401834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4b2a0-be0d-4db4-9a48-0ce5f0e526d0 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:12.422707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:12.468390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:12.502208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa70edb-befc-45c7-9b46-4a257a647921 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:12.528912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4b2a0-be0d-4db4-9a48-0ce5f0e526d0 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:12.542021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76bd71-1fb6-4262-b79b-1f1a05b8d762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:12.542668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76bd71-1fb6-4262-b79b-1f1a05b8d762 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:12.542953 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76bd71-1fb6-4262-b79b-1f1a05b8d762 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:12.543226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a76bd71-1fb6-4262-b79b-1f1a05b8d762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:12.567268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f154afc-a5d6-4aaf-b23e-f45e8186ce0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:12.567885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f154afc-a5d6-4aaf-b23e-f45e8186ce0f None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:12.568160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f154afc-a5d6-4aaf-b23e-f45e8186ce0f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:12.568432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f154afc-a5d6-4aaf-b23e-f45e8186ce0f None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:12.569095 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f154afc-a5d6-4aaf-b23e-f45e8186ce0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:12.570619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f154afc-a5d6-4aaf-b23e-f45e8186ce0f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:12.597936 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 528/1035] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:10:12 2022] POST /identity/v3/users => generated 285 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:10:12.599344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f154afc-a5d6-4aaf-b23e-f45e8186ce0f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:12.648907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a76bd71-1fb6-4262-b79b-1f1a05b8d762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:12.650830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a76bd71-1fb6-4262-b79b-1f1a05b8d762 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:12.700741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:12.737389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f154afc-a5d6-4aaf-b23e-f45e8186ce0f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:12.754602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:12.780360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0324f6-7191-4d21-a174-d701574eb10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:12.782073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0324f6-7191-4d21-a174-d701574eb10d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:12.782073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0324f6-7191-4d21-a174-d701574eb10d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:12.782301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0324f6-7191-4d21-a174-d701574eb10d None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:12.782871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0324f6-7191-4d21-a174-d701574eb10d None admin] RBAC: Adding query filter params (name=image) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:12.783201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0324f6-7191-4d21-a174-d701574eb10d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:12.784608 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0324f6-7191-4d21-a174-d701574eb10d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:12.794540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 529/1036] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:12.798394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0324f6-7191-4d21-a174-d701574eb10d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:12.802262 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 508/1037] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:10:12 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:10:12.809528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76bd71-1fb6-4262-b79b-1f1a05b8d762 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:12.945287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0324f6-7191-4d21-a174-d701574eb10d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:12.949678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a76bd71-1fb6-4262-b79b-1f1a05b8d762 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:12.987542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13919afa-dc2a-4d2c-80f6-0e04b1ee05b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:12.988126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13919afa-dc2a-4d2c-80f6-0e04b1ee05b7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:12.988390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13919afa-dc2a-4d2c-80f6-0e04b1ee05b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:12.988654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13919afa-dc2a-4d2c-80f6-0e04b1ee05b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:12.990897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:13.001765 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13919afa-dc2a-4d2c-80f6-0e04b1ee05b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:13.003250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13919afa-dc2a-4d2c-80f6-0e04b1ee05b7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:13.006639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6bcc6-f6d0-47f9-9982-e21db79fd0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:13.007669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6bcc6-f6d0-47f9-9982-e21db79fd0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:13.008149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6bcc6-f6d0-47f9-9982-e21db79fd0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:13.008612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6bcc6-f6d0-47f9-9982-e21db79fd0b2 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:13.009721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6bcc6-f6d0-47f9-9982-e21db79fd0b2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:13.010262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6bcc6-f6d0-47f9-9982-e21db79fd0b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:13.012839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6bcc6-f6d0-47f9-9982-e21db79fd0b2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-13919afa-dc2a-4d2c-80f6-0e04b1ee05b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:10:13.020709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:13.021777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:10:13.027596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:10:13.027596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:13.027596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:13.027596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 509/1038] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:10:13.027852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 530/1039] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:13.028555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13919afa-dc2a-4d2c-80f6-0e04b1ee05b7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:13.030512 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6bcc6-f6d0-47f9-9982-e21db79fd0b2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:13.103793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:13.167437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13919afa-dc2a-4d2c-80f6-0e04b1ee05b7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:13.178837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6bcc6-f6d0-47f9-9982-e21db79fd0b2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:13.222639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f8e3e-f85d-4327-a973-5e721766f3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:13.223436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f8e3e-f85d-4327-a973-5e721766f3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:13.223879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f8e3e-f85d-4327-a973-5e721766f3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:13.224296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f8e3e-f85d-4327-a973-5e721766f3a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:13.225185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f8e3e-f85d-4327-a973-5e721766f3a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:13.225624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f8e3e-f85d-4327-a973-5e721766f3a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:13.225941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c3120-60b2-450f-b9da-f9e736475e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:13.227046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c3120-60b2-450f-b9da-f9e736475e10 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:13.227536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f8e3e-f85d-4327-a973-5e721766f3a5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:13.227699 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c3120-60b2-450f-b9da-f9e736475e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:13.228047 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c3120-60b2-450f-b9da-f9e736475e10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:13.229456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c3120-60b2-450f-b9da-f9e736475e10 None admin] RBAC: Adding query filter params (interface=public, service_id=bd80b3adb4f44d47bb2a7ecd5525f61a, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:13.229997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c3120-60b2-450f-b9da-f9e736475e10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:13.232752 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c3120-60b2-450f-b9da-f9e736475e10 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:13.243434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 510/1040] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Sep 21 16:10:13 2022] GET /identity/v3/endpoints?service_id=bd80b3adb4f44d47bb2a7ecd5525f61a&interface=public®ion_id=RegionOne => generated 194 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:13.247804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 531/1041] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:13.251170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c3120-60b2-450f-b9da-f9e736475e10 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:13.254103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f8e3e-f85d-4327-a973-5e721766f3a5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:13.311892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:13.370191 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c3120-60b2-450f-b9da-f9e736475e10 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:13.394785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f8e3e-f85d-4327-a973-5e721766f3a5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:13.394785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:13.416685 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ea335-b05d-4b32-bd0e-7c501e6d3f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:13.417307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ea335-b05d-4b32-bd0e-7c501e6d3f41 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:13.417589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ea335-b05d-4b32-bd0e-7c501e6d3f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:13.417866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ea335-b05d-4b32-bd0e-7c501e6d3f41 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:13.418534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ea335-b05d-4b32-bd0e-7c501e6d3f41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:13.420148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ea335-b05d-4b32-bd0e-7c501e6d3f41 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:13.434859 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 511/1042] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Sep 21 16:10:13 2022] GET /identity/v3/services => generated 1957 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:10:13.459080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6862129-37a7-43a8-90f6-29dfc8b6bd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:13.460276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6862129-37a7-43a8-90f6-29dfc8b6bd8c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:13.460768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6862129-37a7-43a8-90f6-29dfc8b6bd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:13.461252 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6862129-37a7-43a8-90f6-29dfc8b6bd8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:13.474807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6862129-37a7-43a8-90f6-29dfc8b6bd8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:13.477642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6862129-37a7-43a8-90f6-29dfc8b6bd8c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a6862129-37a7-43a8-90f6-29dfc8b6bd8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:10:13.502440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:13.503741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:13.514797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:10:13.514797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:10:13.514797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:10:13.514797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:10:13.514797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:10:13.514797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:10:13.514797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:10:13.514797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:13.514797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 532/1043] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:10:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:10:13.518127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ea335-b05d-4b32-bd0e-7c501e6d3f41 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:13.661848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ea335-b05d-4b32-bd0e-7c501e6d3f41 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:13.679712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:13.703322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ec983-7642-482f-a1cc-0b5204275f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:13.703930 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ec983-7642-482f-a1cc-0b5204275f55 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:13.704207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ec983-7642-482f-a1cc-0b5204275f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:13.704473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ec983-7642-482f-a1cc-0b5204275f55 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:13.705165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ec983-7642-482f-a1cc-0b5204275f55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:13.705472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ec983-7642-482f-a1cc-0b5204275f55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:13.707690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ec983-7642-482f-a1cc-0b5204275f55 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:13.726075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 512/1044] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:10:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:13.728208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6862129-37a7-43a8-90f6-29dfc8b6bd8c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:13.846566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6862129-37a7-43a8-90f6-29dfc8b6bd8c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:13.886936 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3fb78-8a0b-47e6-b179-5fe3efcfaa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:13.887583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3fb78-8a0b-47e6-b179-5fe3efcfaa41 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:13.887868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3fb78-8a0b-47e6-b179-5fe3efcfaa41 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:13.888186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3fb78-8a0b-47e6-b179-5fe3efcfaa41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:13.888959 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3fb78-8a0b-47e6-b179-5fe3efcfaa41 None admin] RBAC: Adding query filter params (role.id=8e59f99483d148c09246abf67508a11b, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=f0acf23d224e4b16ad03cdd9bf7cae8e) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:13.889215 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3fb78-8a0b-47e6-b179-5fe3efcfaa41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:13.890676 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3fb78-8a0b-47e6-b179-5fe3efcfaa41 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:13.899771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 533/1045] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:10:13 2022] GET /identity/v3/role_assignments?user.id=f0acf23d224e4b16ad03cdd9bf7cae8e&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=8e59f99483d148c09246abf67508a11b => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:15.138704 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6fb08-decb-45cc-8799-8261dbbf26bd None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:15.139299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6fb08-decb-45cc-8799-8261dbbf26bd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:15.139714 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6fb08-decb-45cc-8799-8261dbbf26bd None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:15.141924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 513/1046] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:15.151785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44372217-759f-4852-b6ad-522962496169 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:15.152320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44372217-759f-4852-b6ad-522962496169 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:15.152766 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44372217-759f-4852-b6ad-522962496169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:15.183945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-44372217-759f-4852-b6ad-522962496169 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:15.185973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-44372217-759f-4852-b6ad-522962496169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:15.222265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:15.285366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 534/1047] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:15.557281 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6fb08-decb-45cc-8799-8261dbbf26bd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:15.682292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6fb08-decb-45cc-8799-8261dbbf26bd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:15.724518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef8b64-3987-431d-9218-32c461646588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:15.725328 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef8b64-3987-431d-9218-32c461646588 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:15.725607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef8b64-3987-431d-9218-32c461646588 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:15.725879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef8b64-3987-431d-9218-32c461646588 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:15.737194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef8b64-3987-431d-9218-32c461646588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-79ef8b64-3987-431d-9218-32c461646588 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:10:15.744018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:15.745346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:15.746923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:15.746923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:15.746923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:15.746923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:15.746923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:10:15.746923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:10:15.746923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:10:15.746923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:10:15.746923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:10:15.746923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:10:15.746923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:10:15.746923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:15.749775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 514/1048] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:10:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:15.753522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44372217-759f-4852-b6ad-522962496169 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:15.758326 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2eb33b-e9cb-4af8-b17a-8e92dd3d4d0b None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:15.758617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2eb33b-e9cb-4af8-b17a-8e92dd3d4d0b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:15.758877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2eb33b-e9cb-4af8-b17a-8e92dd3d4d0b None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:15.761989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 515/1049] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:10:15.767752 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88922075-14ad-4eed-9f4b-680d8af00e1b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:15.768114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88922075-14ad-4eed-9f4b-680d8af00e1b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:15.768360 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88922075-14ad-4eed-9f4b-680d8af00e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:15.794548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-88922075-14ad-4eed-9f4b-680d8af00e1b None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:15.795928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-88922075-14ad-4eed-9f4b-680d8af00e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:15.849269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:15.904349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44372217-759f-4852-b6ad-522962496169 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:15.911841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 516/1050] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:10:15.948215 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf49f8-a20a-452c-8499-4ccb2e4fb2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:15.949128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf49f8-a20a-452c-8499-4ccb2e4fb2bd None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:15.949609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf49f8-a20a-452c-8499-4ccb2e4fb2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:15.950120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf49f8-a20a-452c-8499-4ccb2e4fb2bd None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:15.950994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf49f8-a20a-452c-8499-4ccb2e4fb2bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:15.951485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf49f8-a20a-452c-8499-4ccb2e4fb2bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:15.953029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf49f8-a20a-452c-8499-4ccb2e4fb2bd None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:15.963988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 535/1051] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:10:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:15.971088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88922075-14ad-4eed-9f4b-680d8af00e1b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:16.115485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88922075-14ad-4eed-9f4b-680d8af00e1b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:16.160742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ffdc2-205d-4a6a-9dfc-ea9bc25619d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:16.161449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ffdc2-205d-4a6a-9dfc-ea9bc25619d1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:16.161636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ffdc2-205d-4a6a-9dfc-ea9bc25619d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:16.162027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ffdc2-205d-4a6a-9dfc-ea9bc25619d1 None admin] PATH_INFO: `/v3/users/59f6dd0954a24139b4d9ea72dc852e51` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:16.179383 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ffdc2-205d-4a6a-9dfc-ea9bc25619d1 None admin] RBAC: Authorizing `identity:get_user(user_id=59f6dd0954a24139b4d9ea72dc852e51)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:16.180954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ffdc2-205d-4a6a-9dfc-ea9bc25619d1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:16.183353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 517/1052] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:10:15 2022] GET /identity/v3/users/59f6dd0954a24139b4d9ea72dc852e51 => generated 285 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:16.190393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf49f8-a20a-452c-8499-4ccb2e4fb2bd None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:16.193971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ffdc2-205d-4a6a-9dfc-ea9bc25619d1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:16.227210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:16.228539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:16.332093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ffdc2-205d-4a6a-9dfc-ea9bc25619d1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:16.332443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf49f8-a20a-452c-8499-4ccb2e4fb2bd None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:16.371283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9b520-33e6-45a3-9221-90ca510eb21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:16.371825 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9b520-33e6-45a3-9221-90ca510eb21d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:16.372099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9b520-33e6-45a3-9221-90ca510eb21d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:16.372367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9b520-33e6-45a3-9221-90ca510eb21d None admin] PATH_INFO: `/v3/services/image` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:16.372976 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9b520-33e6-45a3-9221-90ca510eb21d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:16.374442 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9b520-33e6-45a3-9221-90ca510eb21d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1fc9b520-33e6-45a3-9221-90ca510eb21d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:16.384603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:16.385708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:16.386608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 518/1053] 10.222.0.64 () {56 vars in 1105 bytes} [Wed Sep 21 16:10:16 2022] GET /identity/v3/services/image => generated 86 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:10:16.387429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08465a90-58de-4e01-9a7b-889534d8e3b2 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:16.387429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08465a90-58de-4e01-9a7b-889534d8e3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:16.388200 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08465a90-58de-4e01-9a7b-889534d8e3b2 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:16.388636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2197aeb-d82f-45d9-8612-c263e5897854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:16.389209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 519/1054] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:16.389400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2197aeb-d82f-45d9-8612-c263e5897854 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:16.389759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2197aeb-d82f-45d9-8612-c263e5897854 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:16.390166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2197aeb-d82f-45d9-8612-c263e5897854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:16.395408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9529d52-44e6-4aec-ba82-0ddcb70dde9b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:16.395667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9529d52-44e6-4aec-ba82-0ddcb70dde9b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:16.395944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9529d52-44e6-4aec-ba82-0ddcb70dde9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:16.415439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e9529d52-44e6-4aec-ba82-0ddcb70dde9b None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:16.416653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e9529d52-44e6-4aec-ba82-0ddcb70dde9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:16.475536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2197aeb-d82f-45d9-8612-c263e5897854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:16.477293 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2197aeb-d82f-45d9-8612-c263e5897854 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:16.496750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 520/1055] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:10:16.497620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9529d52-44e6-4aec-ba82-0ddcb70dde9b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:16.511285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:16.526917 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:16.641061 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9529d52-44e6-4aec-ba82-0ddcb70dde9b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:16.649221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 536/1056] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:10:16 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:10:16.651246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2197aeb-d82f-45d9-8612-c263e5897854 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:16.680192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa65e2-05b5-45e4-a6cb-aeba0afed72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:16.681215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa65e2-05b5-45e4-a6cb-aeba0afed72d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:16.681696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa65e2-05b5-45e4-a6cb-aeba0afed72d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:16.682216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa65e2-05b5-45e4-a6cb-aeba0afed72d None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:16.683502 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa65e2-05b5-45e4-a6cb-aeba0afed72d None admin] RBAC: Adding query filter params (name=image) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:16.684141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa65e2-05b5-45e4-a6cb-aeba0afed72d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:16.686507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa65e2-05b5-45e4-a6cb-aeba0afed72d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:16.700588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 521/1057] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:16.701884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa65e2-05b5-45e4-a6cb-aeba0afed72d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:16.772946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2197aeb-d82f-45d9-8612-c263e5897854 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:16.792351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:16.799174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:16.812112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c2976-4935-4b34-809a-501f0760fda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:16.812470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c2976-4935-4b34-809a-501f0760fda2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:16.812755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c2976-4935-4b34-809a-501f0760fda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:16.813024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c2976-4935-4b34-809a-501f0760fda2 None admin] PATH_INFO: `/v3/projects/9eb72c15001d4740beb34fac8e10d6b8` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:16.814258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c2976-4935-4b34-809a-501f0760fda2 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb72c15001d4740beb34fac8e10d6b8)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:16.815785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c2976-4935-4b34-809a-501f0760fda2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:16.817553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 537/1058] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:10:16 2022] GET /identity/v3/projects/9eb72c15001d4740beb34fac8e10d6b8 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:16.818428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c2976-4935-4b34-809a-501f0760fda2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:16.822206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa65e2-05b5-45e4-a6cb-aeba0afed72d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:16.862429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679f51d-226f-4d58-a927-93d5e1c1ef91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:16.862429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679f51d-226f-4d58-a927-93d5e1c1ef91 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:16.862429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679f51d-226f-4d58-a927-93d5e1c1ef91 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:16.862429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679f51d-226f-4d58-a927-93d5e1c1ef91 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:16.862901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679f51d-226f-4d58-a927-93d5e1c1ef91 None admin] RBAC: Adding query filter params (type=image) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:16.862901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679f51d-226f-4d58-a927-93d5e1c1ef91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:16.864170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a679f51d-226f-4d58-a927-93d5e1c1ef91 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:16.869489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 522/1059] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:16.870353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679f51d-226f-4d58-a927-93d5e1c1ef91 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:16.931900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c2976-4935-4b34-809a-501f0760fda2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:16.968421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbcca8-1f32-4f17-acfd-e643578ee29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:16.969053 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbcca8-1f32-4f17-acfd-e643578ee29a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:16.969334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbcca8-1f32-4f17-acfd-e643578ee29a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:16.969604 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfbcca8-1f32-4f17-acfd-e643578ee29a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:16.979570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679f51d-226f-4d58-a927-93d5e1c1ef91 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:16.980987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfbcca8-1f32-4f17-acfd-e643578ee29a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-cdfbcca8-1f32-4f17-acfd-e643578ee29a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:10:16.985914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:16.987810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 538/1060] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Sep 21 16:10:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:16.989047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbcca8-1f32-4f17-acfd-e643578ee29a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:17.021620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e27ae-c714-40e4-be39-359fafd5181a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:17.022327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e27ae-c714-40e4-be39-359fafd5181a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:17.022608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e27ae-c714-40e4-be39-359fafd5181a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:17.022877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e27ae-c714-40e4-be39-359fafd5181a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:17.023608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e27ae-c714-40e4-be39-359fafd5181a None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=9eb72c15001d4740beb34fac8e10d6b8, user.id=59f6dd0954a24139b4d9ea72dc852e51) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:17.023917 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e27ae-c714-40e4-be39-359fafd5181a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:17.025349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e27ae-c714-40e4-be39-359fafd5181a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:17.032008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 523/1061] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:10:16 2022] GET /identity/v3/role_assignments?user.id=59f6dd0954a24139b4d9ea72dc852e51&scope.project.id=9eb72c15001d4740beb34fac8e10d6b8&role.id=459d01a218314543a7a8329e88b78331 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:17.032008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e27ae-c714-40e4-be39-359fafd5181a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:17.044291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:17.058694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:17.113265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfbcca8-1f32-4f17-acfd-e643578ee29a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:17.157438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fb24f-854b-4448-8adc-734e78cc96ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:17.158097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fb24f-854b-4448-8adc-734e78cc96ce None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:17.158381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fb24f-854b-4448-8adc-734e78cc96ce None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:17.158652 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fb24f-854b-4448-8adc-734e78cc96ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:17.159888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fb24f-854b-4448-8adc-734e78cc96ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:17.161309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fb24f-854b-4448-8adc-734e78cc96ce None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:17.186413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 539/1062] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:10:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:10:17.235410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e27ae-c714-40e4-be39-359fafd5181a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:17.288291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dfd53-51b7-4f8c-99b4-470f244dbc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:17.288932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dfd53-51b7-4f8c-99b4-470f244dbc47 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:17.289224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dfd53-51b7-4f8c-99b4-470f244dbc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:17.289482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dfd53-51b7-4f8c-99b4-470f244dbc47 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:17.290209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555dfd53-51b7-4f8c-99b4-470f244dbc47 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:17.290519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555dfd53-51b7-4f8c-99b4-470f244dbc47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:17.291957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555dfd53-51b7-4f8c-99b4-470f244dbc47 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:17.309242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 524/1063] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Sep 21 16:10:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:17.315094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fb24f-854b-4448-8adc-734e78cc96ce None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:17.448808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fb24f-854b-4448-8adc-734e78cc96ce None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:17.450421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:17.494410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd23f21-80ec-4244-8785-c1161620f41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:17.495069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd23f21-80ec-4244-8785-c1161620f41f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:17.495349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd23f21-80ec-4244-8785-c1161620f41f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:17.495629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd23f21-80ec-4244-8785-c1161620f41f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:17.513480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd23f21-80ec-4244-8785-c1161620f41f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:17.516185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd23f21-80ec-4244-8785-c1161620f41f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-bfd23f21-80ec-4244-8785-c1161620f41f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:10:17.544699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:17.545956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:10:17.547562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:10:17.547562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:17.547562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:17.547562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 540/1064] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:17.552633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dfd53-51b7-4f8c-99b4-470f244dbc47 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:17.630286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:17.670143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dfd53-51b7-4f8c-99b4-470f244dbc47 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:17.707488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3351af-7614-4374-91d3-173e71b98b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:17.708012 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3351af-7614-4374-91d3-173e71b98b04 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:17.708359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3351af-7614-4374-91d3-173e71b98b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:17.708630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3351af-7614-4374-91d3-173e71b98b04 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:17.709273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3351af-7614-4374-91d3-173e71b98b04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:17.709571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3351af-7614-4374-91d3-173e71b98b04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:17.710946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3351af-7614-4374-91d3-173e71b98b04 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:17.725010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 525/1065] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:17.729682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd23f21-80ec-4244-8785-c1161620f41f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:17.843993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd23f21-80ec-4244-8785-c1161620f41f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:17.887795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477afc6f-b7b9-4b5a-a08e-1e45ffb39ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:17.888443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477afc6f-b7b9-4b5a-a08e-1e45ffb39ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:17.888721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477afc6f-b7b9-4b5a-a08e-1e45ffb39ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:17.888994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477afc6f-b7b9-4b5a-a08e-1e45ffb39ba5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:17.898535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477afc6f-b7b9-4b5a-a08e-1e45ffb39ba5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:17.899984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477afc6f-b7b9-4b5a-a08e-1e45ffb39ba5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-477afc6f-b7b9-4b5a-a08e-1e45ffb39ba5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:10:17.914579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:17.915639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:17.916592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:17.916592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:17.916592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:10:17.916592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:10:17.916592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:10:17.916592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:10:17.916592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:10:17.916592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:10:17.916592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:10:17.916592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:17.918130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 541/1066] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Sep 21 16:10:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:17.922671 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3351af-7614-4374-91d3-173e71b98b04 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:18.052093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3351af-7614-4374-91d3-173e71b98b04 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:18.085414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:18.095686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce225f-8f9a-4176-809a-24cb9536585b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:18.096816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce225f-8f9a-4176-809a-24cb9536585b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:18.096816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce225f-8f9a-4176-809a-24cb9536585b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:18.097734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce225f-8f9a-4176-809a-24cb9536585b None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:18.098298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce225f-8f9a-4176-809a-24cb9536585b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:18.098669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce225f-8f9a-4176-809a-24cb9536585b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:18.100142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ce225f-8f9a-4176-809a-24cb9536585b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:18.126151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 526/1067] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Sep 21 16:10:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:18.127450 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477afc6f-b7b9-4b5a-a08e-1e45ffb39ba5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:18.153356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:18.254188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477afc6f-b7b9-4b5a-a08e-1e45ffb39ba5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:18.297697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c623f-a3a5-4bdf-8797-c5ef9b1fea26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:18.298381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c623f-a3a5-4bdf-8797-c5ef9b1fea26 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:18.298661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c623f-a3a5-4bdf-8797-c5ef9b1fea26 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:18.299049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c623f-a3a5-4bdf-8797-c5ef9b1fea26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:18.311866 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c623f-a3a5-4bdf-8797-c5ef9b1fea26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:18.313401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c623f-a3a5-4bdf-8797-c5ef9b1fea26 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e51c623f-a3a5-4bdf-8797-c5ef9b1fea26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:10:18.332447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:18.334035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:10:18.335114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:10:18.335114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:18.335114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:18.335114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 542/1068] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:18.342163 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce225f-8f9a-4176-809a-24cb9536585b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:18.466288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce225f-8f9a-4176-809a-24cb9536585b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:18.517108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d00523-8ffb-460f-86a9-9feeefce2c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:18.517735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d00523-8ffb-460f-86a9-9feeefce2c9f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:18.518014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d00523-8ffb-460f-86a9-9feeefce2c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:18.518444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d00523-8ffb-460f-86a9-9feeefce2c9f None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:18.519674 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d00523-8ffb-460f-86a9-9feeefce2c9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:18.519974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d00523-8ffb-460f-86a9-9feeefce2c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:18.521340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d00523-8ffb-460f-86a9-9feeefce2c9f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:18.538463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 527/1069] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:18.539824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c623f-a3a5-4bdf-8797-c5ef9b1fea26 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:18.617796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:18.664417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c623f-a3a5-4bdf-8797-c5ef9b1fea26 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:18.705548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaa038-3890-4bbd-acf9-225b828af5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:18.706810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaa038-3890-4bbd-acf9-225b828af5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:18.707091 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaa038-3890-4bbd-acf9-225b828af5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:18.707364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaa038-3890-4bbd-acf9-225b828af5a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:18.716712 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcaa038-3890-4bbd-acf9-225b828af5a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:18.718323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcaa038-3890-4bbd-acf9-225b828af5a6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6dcaa038-3890-4bbd-acf9-225b828af5a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:10:18.730639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:18.731620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:18.732515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:10:18.732515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:10:18.732515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:10:18.732515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:10:18.732515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:10:18.732515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:10:18.732515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:10:18.732515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:18.734140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 543/1070] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:10:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:18.738352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d00523-8ffb-460f-86a9-9feeefce2c9f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:18.792100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:18.861624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d00523-8ffb-460f-86a9-9feeefce2c9f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:18.909144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7e91c-5226-4fc4-a20e-5e2b31fe67da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:18.909790 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7e91c-5226-4fc4-a20e-5e2b31fe67da None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:18.910096 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7e91c-5226-4fc4-a20e-5e2b31fe67da None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:18.910377 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7e91c-5226-4fc4-a20e-5e2b31fe67da None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:18.911087 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7e91c-5226-4fc4-a20e-5e2b31fe67da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:18.911390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7e91c-5226-4fc4-a20e-5e2b31fe67da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:18.912846 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7e91c-5226-4fc4-a20e-5e2b31fe67da None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:18.927503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 528/1071] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:10:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:18.933171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaa038-3890-4bbd-acf9-225b828af5a6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:19.041614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaa038-3890-4bbd-acf9-225b828af5a6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:19.071343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:19.081803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d090dc2-3608-43d5-b07b-0fd5b4cf06e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:19.082501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d090dc2-3608-43d5-b07b-0fd5b4cf06e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:19.082780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d090dc2-3608-43d5-b07b-0fd5b4cf06e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:19.083052 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d090dc2-3608-43d5-b07b-0fd5b4cf06e1 None admin] PATH_INFO: `/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/f0acf23d224e4b16ad03cdd9bf7cae8e/roles/8e59f99483d148c09246abf67508a11b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:19.085652 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d090dc2-3608-43d5-b07b-0fd5b4cf06e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9e1d9331b4d4dbd3dd447937c42ad, user_id=f0acf23d224e4b16ad03cdd9bf7cae8e, role_id=8e59f99483d148c09246abf67508a11b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:19.087113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d090dc2-3608-43d5-b07b-0fd5b4cf06e1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:19.102585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 544/1072] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:10:18 2022] PUT /identity/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/f0acf23d224e4b16ad03cdd9bf7cae8e/roles/8e59f99483d148c09246abf67508a11b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:10:19.253581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab6138-4e1a-4a2b-a4b0-1e4d0f1581ad None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:19.253776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab6138-4e1a-4a2b-a4b0-1e4d0f1581ad None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:19.254082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab6138-4e1a-4a2b-a4b0-1e4d0f1581ad None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:19.255537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 529/1073] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:19.261680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb1257-7d75-4285-804b-b779c9d785fa None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:19.261983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb1257-7d75-4285-804b-b779c9d785fa None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:19.262276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb1257-7d75-4285-804b-b779c9d785fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:19.283103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d8fb1257-7d75-4285-804b-b779c9d785fa None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:19.284650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d8fb1257-7d75-4285-804b-b779c9d785fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:19.363667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 545/1074] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:19.450738 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dc8a6-03ac-4b69-a8ad-0b943dc235a4 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:19.451330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dc8a6-03ac-4b69-a8ad-0b943dc235a4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:19.451751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639dc8a6-03ac-4b69-a8ad-0b943dc235a4 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:19.454583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 530/1075] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:19.460477 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3662f6f-f872-4131-8a53-827987d7941c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:19.460773 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3662f6f-f872-4131-8a53-827987d7941c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:19.461049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3662f6f-f872-4131-8a53-827987d7941c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:19.479805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a3662f6f-f872-4131-8a53-827987d7941c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:19.483531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a3662f6f-f872-4131-8a53-827987d7941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:19.574445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 546/1076] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:19.638282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dc8a6-03ac-4b69-a8ad-0b943dc235a4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:19.754259 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639dc8a6-03ac-4b69-a8ad-0b943dc235a4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:19.756694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:19.798925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa7713-857d-4c12-bfbf-516b5f928654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:19.799629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa7713-857d-4c12-bfbf-516b5f928654 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:19.799969 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa7713-857d-4c12-bfbf-516b5f928654 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:19.800406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa7713-857d-4c12-bfbf-516b5f928654 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:19.812219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa7713-857d-4c12-bfbf-516b5f928654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-80aa7713-857d-4c12-bfbf-516b5f928654 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:10:19.817113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:19.818210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:19.819172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 531/1077] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:10:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:19.824915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3662f6f-f872-4131-8a53-827987d7941c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:19.840182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa7713-857d-4c12-bfbf-516b5f928654 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:19.877756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:19.953654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3662f6f-f872-4131-8a53-827987d7941c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:19.958445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa7713-857d-4c12-bfbf-516b5f928654 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:19.995905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761efca4-1ecc-4add-a724-72f79a0ff024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:19.996706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761efca4-1ecc-4add-a724-72f79a0ff024 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:19.996991 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761efca4-1ecc-4add-a724-72f79a0ff024 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:19.997265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761efca4-1ecc-4add-a724-72f79a0ff024 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:20.014454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5453991-2656-4ae3-834d-4f0a54480715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:20.015086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5453991-2656-4ae3-834d-4f0a54480715 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:20.015359 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5453991-2656-4ae3-834d-4f0a54480715 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:20.015659 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5453991-2656-4ae3-834d-4f0a54480715 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:20.016327 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5453991-2656-4ae3-834d-4f0a54480715 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:20.016638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5453991-2656-4ae3-834d-4f0a54480715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:20.018105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5453991-2656-4ae3-834d-4f0a54480715 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:20.036004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 547/1078] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:10:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:20.037639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5453991-2656-4ae3-834d-4f0a54480715 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:20.047306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:20.095515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761efca4-1ecc-4add-a724-72f79a0ff024 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:20.097455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761efca4-1ecc-4add-a724-72f79a0ff024 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:20.179746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5453991-2656-4ae3-834d-4f0a54480715 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:20.191863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:20.222881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a8ad2-497d-464a-bea6-39c2b099e88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:20.223613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a8ad2-497d-464a-bea6-39c2b099e88d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:20.223842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a8ad2-497d-464a-bea6-39c2b099e88d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:20.226123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a8ad2-497d-464a-bea6-39c2b099e88d None admin] PATH_INFO: `/v3/users/59f6dd0954a24139b4d9ea72dc852e51` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:20.226264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 532/1079] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:10:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 16:10:20.226371 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a8ad2-497d-464a-bea6-39c2b099e88d None admin] RBAC: Authorizing `identity:get_user(user_id=59f6dd0954a24139b4d9ea72dc852e51)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:20.226716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a8ad2-497d-464a-bea6-39c2b099e88d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:20.230238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 548/1080] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:10:20 2022] GET /identity/v3/users/59f6dd0954a24139b4d9ea72dc852e51 => generated 285 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:20.231048 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761efca4-1ecc-4add-a724-72f79a0ff024 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:20.234372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a8ad2-497d-464a-bea6-39c2b099e88d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:20.300820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:20.355276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761efca4-1ecc-4add-a724-72f79a0ff024 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:20.359492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a8ad2-497d-464a-bea6-39c2b099e88d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:20.399282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceec06-2446-47ab-9e25-61c3236798af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:20.399967 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098e97e-7b2f-4f7f-902c-202b6697be5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:20.400103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceec06-2446-47ab-9e25-61c3236798af None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:20.400204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceec06-2446-47ab-9e25-61c3236798af None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:20.400467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceec06-2446-47ab-9e25-61c3236798af None admin] PATH_INFO: `/v3/projects/9eb72c15001d4740beb34fac8e10d6b8` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:20.400781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098e97e-7b2f-4f7f-902c-202b6697be5d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:20.401180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098e97e-7b2f-4f7f-902c-202b6697be5d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:20.401603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098e97e-7b2f-4f7f-902c-202b6697be5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:20.401699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceec06-2446-47ab-9e25-61c3236798af None admin] RBAC: Authorizing `identity:get_project(project_id=9eb72c15001d4740beb34fac8e10d6b8)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:20.403069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceec06-2446-47ab-9e25-61c3236798af None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:20.404761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 549/1081] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:10:20 2022] GET /identity/v3/projects/9eb72c15001d4740beb34fac8e10d6b8 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:20.410931 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceec06-2446-47ab-9e25-61c3236798af None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:20.420113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098e97e-7b2f-4f7f-902c-202b6697be5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:20.421927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098e97e-7b2f-4f7f-902c-202b6697be5d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1098e97e-7b2f-4f7f-902c-202b6697be5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:10:20.450336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:20.451833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:10:20.452727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:10:20.452727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:20.452727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:20.452727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 533/1082] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:20.459169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098e97e-7b2f-4f7f-902c-202b6697be5d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:20.492606 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:20.570306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceec06-2446-47ab-9e25-61c3236798af None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:20.608222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1688ff7-b88d-4d1e-9010-1c4cdc13466b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:20.608952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1688ff7-b88d-4d1e-9010-1c4cdc13466b None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:20.609326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1688ff7-b88d-4d1e-9010-1c4cdc13466b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:20.609693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1688ff7-b88d-4d1e-9010-1c4cdc13466b None admin] PATH_INFO: `/v3/projects/9eb72c15001d4740beb34fac8e10d6b8/users/59f6dd0954a24139b4d9ea72dc852e51/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:20.611756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1688ff7-b88d-4d1e-9010-1c4cdc13466b None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb72c15001d4740beb34fac8e10d6b8, user_id=59f6dd0954a24139b4d9ea72dc852e51, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:20.614434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1688ff7-b88d-4d1e-9010-1c4cdc13466b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:20.632323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 550/1083] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:10:20 2022] PUT /identity/v3/projects/9eb72c15001d4740beb34fac8e10d6b8/users/59f6dd0954a24139b4d9ea72dc852e51/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:10:20.634278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098e97e-7b2f-4f7f-902c-202b6697be5d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:20.673032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:20.709072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2a023-3764-4569-8e85-18949e92dfc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:20.710206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2a023-3764-4569-8e85-18949e92dfc6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:20.710709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2a023-3764-4569-8e85-18949e92dfc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:20.711183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2a023-3764-4569-8e85-18949e92dfc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:20.712593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2a023-3764-4569-8e85-18949e92dfc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:20.712753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2a023-3764-4569-8e85-18949e92dfc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:20.715462 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2a023-3764-4569-8e85-18949e92dfc6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:20.734839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 534/1084] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:21.410468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2e7ca-e2d2-427b-9b4a-97e98bf5e33e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:21.411175 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2e7ca-e2d2-427b-9b4a-97e98bf5e33e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:21.411175 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2e7ca-e2d2-427b-9b4a-97e98bf5e33e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:21.412366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 551/1085] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:21.418173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206db18-439b-400e-84e0-880b62244334 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:21.418467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206db18-439b-400e-84e0-880b62244334 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:21.418853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3206db18-439b-400e-84e0-880b62244334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:21.440091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3206db18-439b-400e-84e0-880b62244334 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:21.441813 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3206db18-439b-400e-84e0-880b62244334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:21.525669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 535/1086] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:21.799473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2e7ca-e2d2-427b-9b4a-97e98bf5e33e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:21.943891 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2e7ca-e2d2-427b-9b4a-97e98bf5e33e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:21.947288 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:22.005967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c64c7-5ae1-42e5-b190-7d1827fdae2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:22.007041 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c64c7-5ae1-42e5-b190-7d1827fdae2b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:22.007472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c64c7-5ae1-42e5-b190-7d1827fdae2b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:22.007898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c64c7-5ae1-42e5-b190-7d1827fdae2b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:22.026227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c64c7-5ae1-42e5-b190-7d1827fdae2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-ea2c64c7-5ae1-42e5-b190-7d1827fdae2b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:10:22.035001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:22.036014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:22.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:22.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:22.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:22.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:22.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:10:22.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:10:22.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:10:22.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:10:22.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:10:22.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:10:22.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 21 16:10:22.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:22.037540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 552/1087] 10.222.0.64 () {56 vars in 1120 bytes} [Wed Sep 21 16:10:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:10:22.045790 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206db18-439b-400e-84e0-880b62244334 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:22.151966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3206db18-439b-400e-84e0-880b62244334 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:22.159963 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:22.197112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2039fe-0216-4a97-a701-f4ee2b37724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:22.197716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2039fe-0216-4a97-a701-f4ee2b37724b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:22.197997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2039fe-0216-4a97-a701-f4ee2b37724b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:22.198303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2039fe-0216-4a97-a701-f4ee2b37724b None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:22.198977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2039fe-0216-4a97-a701-f4ee2b37724b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:22.199279 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2039fe-0216-4a97-a701-f4ee2b37724b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:22.200635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2039fe-0216-4a97-a701-f4ee2b37724b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:22.208771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 536/1088] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Sep 21 16:10:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:22.214640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c64c7-5ae1-42e5-b190-7d1827fdae2b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:22.315471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c64c7-5ae1-42e5-b190-7d1827fdae2b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:22.350661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58b930-a35b-4b59-935a-a3083771664f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:22.351282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58b930-a35b-4b59-935a-a3083771664f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:22.351555 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58b930-a35b-4b59-935a-a3083771664f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:22.351820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58b930-a35b-4b59-935a-a3083771664f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:22.362128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58b930-a35b-4b59-935a-a3083771664f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:22.364067 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58b930-a35b-4b59-935a-a3083771664f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-2c58b930-a35b-4b59-935a-a3083771664f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:10:22.378945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:22.380109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:10:22.381010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:10:22.381010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:22.381010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:22.381010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 553/1089] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:22.386960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2039fe-0216-4a97-a701-f4ee2b37724b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:22.496080 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2039fe-0216-4a97-a701-f4ee2b37724b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:22.530080 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73a1ac-bb2b-4643-b2b9-ad1529ee5fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:22.530770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73a1ac-bb2b-4643-b2b9-ad1529ee5fcc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:22.530946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73a1ac-bb2b-4643-b2b9-ad1529ee5fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:22.531214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73a1ac-bb2b-4643-b2b9-ad1529ee5fcc None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:22.531855 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73a1ac-bb2b-4643-b2b9-ad1529ee5fcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:22.532155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73a1ac-bb2b-4643-b2b9-ad1529ee5fcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:22.533744 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73a1ac-bb2b-4643-b2b9-ad1529ee5fcc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:22.546849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 537/1090] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:22.552517 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58b930-a35b-4b59-935a-a3083771664f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:22.571426 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:22.654734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58b930-a35b-4b59-935a-a3083771664f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:22.688652 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460789e6-d312-4536-b8ed-8d38d67a02a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:22.689282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460789e6-d312-4536-b8ed-8d38d67a02a3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:22.689559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460789e6-d312-4536-b8ed-8d38d67a02a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:22.689828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460789e6-d312-4536-b8ed-8d38d67a02a3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:22.697789 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460789e6-d312-4536-b8ed-8d38d67a02a3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:22.699258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460789e6-d312-4536-b8ed-8d38d67a02a3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-460789e6-d312-4536-b8ed-8d38d67a02a3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:10:22.710972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:22.712163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:22.713207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:22.713207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:22.713207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:10:22.713207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:10:22.713207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:10:22.713207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:10:22.713207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:10:22.713207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:10:22.713207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 21 16:10:22.713207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:22.714200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 554/1091] 10.222.0.64 () {56 vars in 1093 bytes} [Wed Sep 21 16:10:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:22.717753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73a1ac-bb2b-4643-b2b9-ad1529ee5fcc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:22.747034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:22.825626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73a1ac-bb2b-4643-b2b9-ad1529ee5fcc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:22.862974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f7b72-1564-4570-8cef-36d31d52645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:22.863580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f7b72-1564-4570-8cef-36d31d52645c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:22.863856 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f7b72-1564-4570-8cef-36d31d52645c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:22.864126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f7b72-1564-4570-8cef-36d31d52645c None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:22.864852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f7b72-1564-4570-8cef-36d31d52645c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:22.865149 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f7b72-1564-4570-8cef-36d31d52645c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:22.866600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f7b72-1564-4570-8cef-36d31d52645c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:22.884554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 538/1092] 10.222.0.64 () {56 vars in 1133 bytes} [Wed Sep 21 16:10:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:22.890341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460789e6-d312-4536-b8ed-8d38d67a02a3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:22.940880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99363e-5c63-4384-aa7c-ce6a1ec6ed71 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:22.941161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99363e-5c63-4384-aa7c-ce6a1ec6ed71 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:22.941440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99363e-5c63-4384-aa7c-ce6a1ec6ed71 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:22.942809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 539/1093] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:10:22.948262 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5347a-4ee1-42aa-929b-a6a1aa427cdf None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:22.948550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5347a-4ee1-42aa-929b-a6a1aa427cdf None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:22.948822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b5347a-4ee1-42aa-929b-a6a1aa427cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:22.967548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-17b5347a-4ee1-42aa-929b-a6a1aa427cdf None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:22.967866 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:22.968808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-17b5347a-4ee1-42aa-929b-a6a1aa427cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:22.991415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460789e6-d312-4536-b8ed-8d38d67a02a3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:23.026583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a28fa-ca1e-42f7-b3cb-b9deb3a98456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:23.027108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a28fa-ca1e-42f7-b3cb-b9deb3a98456 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:23.027388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a28fa-ca1e-42f7-b3cb-b9deb3a98456 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:23.027653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310a28fa-ca1e-42f7-b3cb-b9deb3a98456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:23.040089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 540/1094] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:10:23.093723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a28fa-ca1e-42f7-b3cb-b9deb3a98456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:23.095196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310a28fa-ca1e-42f7-b3cb-b9deb3a98456 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:23.168620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:23.192345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 555/1095] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:10:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:10:23.198583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5347a-4ee1-42aa-929b-a6a1aa427cdf None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:23.216641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:23.296432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b5347a-4ee1-42aa-929b-a6a1aa427cdf None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:23.310926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a28fa-ca1e-42f7-b3cb-b9deb3a98456 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:23.331558 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f55ad6-e718-478a-80c5-ef0c9c433eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:23.332188 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f55ad6-e718-478a-80c5-ef0c9c433eeb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:23.332463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f55ad6-e718-478a-80c5-ef0c9c433eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:23.332735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f55ad6-e718-478a-80c5-ef0c9c433eeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:23.342939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f55ad6-e718-478a-80c5-ef0c9c433eeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:23.344435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f55ad6-e718-478a-80c5-ef0c9c433eeb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a8f55ad6-e718-478a-80c5-ef0c9c433eeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:10:23.359374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:23.360589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:10:23.361509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:10:23.361509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:10:23.361509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:23.361509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 541/1096] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:23.366989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f55ad6-e718-478a-80c5-ef0c9c433eeb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:23.412376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310a28fa-ca1e-42f7-b3cb-b9deb3a98456 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:23.438878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:23.448474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071d679-c9e0-4d1c-9575-de954befe406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:23.449112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071d679-c9e0-4d1c-9575-de954befe406 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:23.449395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071d679-c9e0-4d1c-9575-de954befe406 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:23.449667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071d679-c9e0-4d1c-9575-de954befe406 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:23.459978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071d679-c9e0-4d1c-9575-de954befe406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-4071d679-c9e0-4d1c-9575-de954befe406 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:10:23.464835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:23.465964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:23.466895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 556/1097] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:10:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:10:23.474460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f55ad6-e718-478a-80c5-ef0c9c433eeb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:23.475788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071d679-c9e0-4d1c-9575-de954befe406 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:23.517622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:23.520326 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0ad4f-9e18-4f10-aa27-b96251416dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:23.520966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0ad4f-9e18-4f10-aa27-b96251416dce None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:23.521248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0ad4f-9e18-4f10-aa27-b96251416dce None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:23.521526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0ad4f-9e18-4f10-aa27-b96251416dce None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:23.522239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0ad4f-9e18-4f10-aa27-b96251416dce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:23.522550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0ad4f-9e18-4f10-aa27-b96251416dce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:23.523918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0ad4f-9e18-4f10-aa27-b96251416dce None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:23.537026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 542/1098] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:23.542586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0ad4f-9e18-4f10-aa27-b96251416dce None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:23.585109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071d679-c9e0-4d1c-9575-de954befe406 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:23.621331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399eff95-b228-4030-94e4-01b07d913627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:23.621941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eff95-b228-4030-94e4-01b07d913627 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:23.622246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eff95-b228-4030-94e4-01b07d913627 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:23.622513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eff95-b228-4030-94e4-01b07d913627 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:23.623179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399eff95-b228-4030-94e4-01b07d913627 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:23.623480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399eff95-b228-4030-94e4-01b07d913627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:23.624826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399eff95-b228-4030-94e4-01b07d913627 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:23.633852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 557/1099] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:10:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:23.640215 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399eff95-b228-4030-94e4-01b07d913627 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:23.657744 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0ad4f-9e18-4f10-aa27-b96251416dce None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:23.686724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:23.696158 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e724f84-1068-444e-93e3-dcdcb2f0d3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:23.696785 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724f84-1068-444e-93e3-dcdcb2f0d3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:23.697069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724f84-1068-444e-93e3-dcdcb2f0d3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:23.697341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e724f84-1068-444e-93e3-dcdcb2f0d3d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:23.704325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e724f84-1068-444e-93e3-dcdcb2f0d3d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:23.705827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e724f84-1068-444e-93e3-dcdcb2f0d3d4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1e724f84-1068-444e-93e3-dcdcb2f0d3d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:10:23.717430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:23.718612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:23.719631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:10:23.719631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:10:23.719631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:10:23.719631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:10:23.719631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:10:23.719631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:10:23.719631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:10:23.719631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:23.719631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 543/1100] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:10:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:10:23.724990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e724f84-1068-444e-93e3-dcdcb2f0d3d4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:23.740043 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:23.744090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399eff95-b228-4030-94e4-01b07d913627 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:23.778442 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254dc3e-0fd4-4d83-b22c-4ed28c439ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:23.779091 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254dc3e-0fd4-4d83-b22c-4ed28c439ade None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:23.779372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254dc3e-0fd4-4d83-b22c-4ed28c439ade None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:23.779641 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254dc3e-0fd4-4d83-b22c-4ed28c439ade None admin] PATH_INFO: `/v3/users/59f6dd0954a24139b4d9ea72dc852e51` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:23.780921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3254dc3e-0fd4-4d83-b22c-4ed28c439ade None admin] RBAC: Authorizing `identity:get_user(user_id=59f6dd0954a24139b4d9ea72dc852e51)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:23.782385 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3254dc3e-0fd4-4d83-b22c-4ed28c439ade None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:23.784520 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 558/1101] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:10:23 2022] GET /identity/v3/users/59f6dd0954a24139b4d9ea72dc852e51 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:23.790222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254dc3e-0fd4-4d83-b22c-4ed28c439ade None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:23.836656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e724f84-1068-444e-93e3-dcdcb2f0d3d4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:23.874166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5ef67-168a-49e6-ba95-4adc40dd5e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:23.874783 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5ef67-168a-49e6-ba95-4adc40dd5e7f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:23.875065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5ef67-168a-49e6-ba95-4adc40dd5e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:23.875340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f5ef67-168a-49e6-ba95-4adc40dd5e7f None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:23.876033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5ef67-168a-49e6-ba95-4adc40dd5e7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:23.876344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5ef67-168a-49e6-ba95-4adc40dd5e7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:23.877778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f5ef67-168a-49e6-ba95-4adc40dd5e7f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:23.890261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 544/1102] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:10:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:23.896073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5ef67-168a-49e6-ba95-4adc40dd5e7f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:23.897797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254dc3e-0fd4-4d83-b22c-4ed28c439ade None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:23.933200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a1a1b-43bd-4882-b413-87edd0aa5966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:23.934087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a1a1b-43bd-4882-b413-87edd0aa5966 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:23.934527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a1a1b-43bd-4882-b413-87edd0aa5966 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:23.934796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a1a1b-43bd-4882-b413-87edd0aa5966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:23.946741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:23.969290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:24.005778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f5ef67-168a-49e6-ba95-4adc40dd5e7f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:24.009774 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a1a1b-43bd-4882-b413-87edd0aa5966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:24.011251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a1a1b-43bd-4882-b413-87edd0aa5966 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:24.040777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943652d7-06df-43df-8c98-9bbd77c5ef77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:24.041386 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943652d7-06df-43df-8c98-9bbd77c5ef77 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:24.041670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943652d7-06df-43df-8c98-9bbd77c5ef77 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:24.041944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943652d7-06df-43df-8c98-9bbd77c5ef77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:24.042704 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943652d7-06df-43df-8c98-9bbd77c5ef77 None admin] RBAC: Adding query filter params (role.id=8e59f99483d148c09246abf67508a11b, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=f0acf23d224e4b16ad03cdd9bf7cae8e) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:24.043013 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943652d7-06df-43df-8c98-9bbd77c5ef77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:24.044445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943652d7-06df-43df-8c98-9bbd77c5ef77 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:24.051321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 545/1103] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:10:23 2022] GET /identity/v3/role_assignments?user.id=f0acf23d224e4b16ad03cdd9bf7cae8e&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=8e59f99483d148c09246abf67508a11b => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:24.122413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 559/1104] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:10:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 16:10:24.129145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943652d7-06df-43df-8c98-9bbd77c5ef77 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:24.248581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943652d7-06df-43df-8c98-9bbd77c5ef77 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:24.263924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:24.287285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b831a48c-587c-4d7e-a4eb-8bfd2a9d9e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:24.287941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831a48c-587c-4d7e-a4eb-8bfd2a9d9e40 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:24.288221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831a48c-587c-4d7e-a4eb-8bfd2a9d9e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:24.288497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b831a48c-587c-4d7e-a4eb-8bfd2a9d9e40 None admin] PATH_INFO: `/v3/projects/9eb72c15001d4740beb34fac8e10d6b8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:24.289634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831a48c-587c-4d7e-a4eb-8bfd2a9d9e40 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb72c15001d4740beb34fac8e10d6b8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:24.291113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b831a48c-587c-4d7e-a4eb-8bfd2a9d9e40 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:24.292773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 546/1105] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:10:24 2022] GET /identity/v3/projects/9eb72c15001d4740beb34fac8e10d6b8 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:24.299969 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a1a1b-43bd-4882-b413-87edd0aa5966 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:24.372663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:24.414627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a1a1b-43bd-4882-b413-87edd0aa5966 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:24.457493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6ccea-c580-462e-94b0-2721207f7ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:24.458145 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6ccea-c580-462e-94b0-2721207f7ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:24.458425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6ccea-c580-462e-94b0-2721207f7ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:24.458705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6ccea-c580-462e-94b0-2721207f7ec0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:24.459478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6ccea-c580-462e-94b0-2721207f7ec0 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=9eb72c15001d4740beb34fac8e10d6b8, user.id=59f6dd0954a24139b4d9ea72dc852e51) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:24.459786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6ccea-c580-462e-94b0-2721207f7ec0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:24.461222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6ccea-c580-462e-94b0-2721207f7ec0 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:24.467602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 560/1106] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:10:24 2022] GET /identity/v3/role_assignments?user.id=59f6dd0954a24139b4d9ea72dc852e51&scope.project.id=9eb72c15001d4740beb34fac8e10d6b8&role.id=459d01a218314543a7a8329e88b78331 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:26.709424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b311691-2ee7-4aec-8ead-f95683671d9a None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:26.709980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b311691-2ee7-4aec-8ead-f95683671d9a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:26.709980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b311691-2ee7-4aec-8ead-f95683671d9a None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:26.711370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 547/1107] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:26.716979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2c1fe-040b-4c6b-96cc-68ae121e089b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:26.717405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2c1fe-040b-4c6b-96cc-68ae121e089b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:26.717515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2c1fe-040b-4c6b-96cc-68ae121e089b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:26.738315 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-05e2c1fe-040b-4c6b-96cc-68ae121e089b None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:26.739422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-05e2c1fe-040b-4c6b-96cc-68ae121e089b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:26.805461 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 561/1108] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:27.081424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b311691-2ee7-4aec-8ead-f95683671d9a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:27.184328 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b311691-2ee7-4aec-8ead-f95683671d9a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:27.218230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6307d1b-4851-480f-ac37-0a20554da449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:27.218966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6307d1b-4851-480f-ac37-0a20554da449 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:27.219243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6307d1b-4851-480f-ac37-0a20554da449 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:27.219510 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6307d1b-4851-480f-ac37-0a20554da449 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:27.220625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6307d1b-4851-480f-ac37-0a20554da449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:27.222092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6307d1b-4851-480f-ac37-0a20554da449 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:27.223662 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 548/1109] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:27.229692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2c1fe-040b-4c6b-96cc-68ae121e089b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:27.288661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:27.329978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2c1fe-040b-4c6b-96cc-68ae121e089b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:27.363654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db07a9-d7fd-4e17-9d83-f1ec60f0193d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:27.364178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db07a9-d7fd-4e17-9d83-f1ec60f0193d None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:27.364450 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db07a9-d7fd-4e17-9d83-f1ec60f0193d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:27.364716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db07a9-d7fd-4e17-9d83-f1ec60f0193d None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:27.365352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db07a9-d7fd-4e17-9d83-f1ec60f0193d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:27.366784 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db07a9-d7fd-4e17-9d83-f1ec60f0193d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:27.386791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 562/1110] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:10:27 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:10:29.581778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596b6dd-60f5-47db-b2e4-14f08de56d94 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:29.582365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596b6dd-60f5-47db-b2e4-14f08de56d94 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:29.582365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596b6dd-60f5-47db-b2e4-14f08de56d94 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:29.583696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 549/1111] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:29.589083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d9a03-6a5f-4f73-869f-de761fbdb103 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:29.589482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d9a03-6a5f-4f73-869f-de761fbdb103 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:29.589621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d9a03-6a5f-4f73-869f-de761fbdb103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:29.607772 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-348d9a03-6a5f-4f73-869f-de761fbdb103 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:29.608931 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-348d9a03-6a5f-4f73-869f-de761fbdb103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:29.672626 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 563/1112] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:29.951864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596b6dd-60f5-47db-b2e4-14f08de56d94 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:30.001931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:30.054250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596b6dd-60f5-47db-b2e4-14f08de56d94 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:30.086742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088f90a-f8ce-497f-9819-ed4341cb1426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:30.087349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088f90a-f8ce-497f-9819-ed4341cb1426 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:30.087621 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088f90a-f8ce-497f-9819-ed4341cb1426 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:30.087885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088f90a-f8ce-497f-9819-ed4341cb1426 None admin] PATH_INFO: `/v3/roles/aa320f864d914f65bebe5463994ff536` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:30.089235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088f90a-f8ce-497f-9819-ed4341cb1426 None admin] RBAC: Authorizing `identity:get_role(role_id=aa320f864d914f65bebe5463994ff536)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:30.090613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088f90a-f8ce-497f-9819-ed4341cb1426 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:30.091902 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 550/1113] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:10:29 2022] GET /identity/v3/roles/aa320f864d914f65bebe5463994ff536 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:30.098369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d9a03-6a5f-4f73-869f-de761fbdb103 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:30.144787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:30.196801 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d9a03-6a5f-4f73-869f-de761fbdb103 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:30.236002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d265a3-13b6-4286-b6b8-01cfb52a7d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:30.236588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d265a3-13b6-4286-b6b8-01cfb52a7d4f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:30.236864 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d265a3-13b6-4286-b6b8-01cfb52a7d4f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:30.237134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d265a3-13b6-4286-b6b8-01cfb52a7d4f None admin] PATH_INFO: `/v3/users/1f59402ba955493e861c25a0ef67e0b7` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:30.250092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d265a3-13b6-4286-b6b8-01cfb52a7d4f None admin] RBAC: Authorizing `identity:get_user(user_id=1f59402ba955493e861c25a0ef67e0b7)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:30.251506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d265a3-13b6-4286-b6b8-01cfb52a7d4f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:30.253213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 564/1114] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:10:30 2022] GET /identity/v3/users/1f59402ba955493e861c25a0ef67e0b7 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:30.259194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088f90a-f8ce-497f-9819-ed4341cb1426 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:30.356176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088f90a-f8ce-497f-9819-ed4341cb1426 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:30.372077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:30.389661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72478c-bc67-4395-aafd-e411384c40dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:30.390284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72478c-bc67-4395-aafd-e411384c40dc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:30.390589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72478c-bc67-4395-aafd-e411384c40dc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:30.390856 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72478c-bc67-4395-aafd-e411384c40dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:30.456221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72478c-bc67-4395-aafd-e411384c40dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:30.457654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72478c-bc67-4395-aafd-e411384c40dc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:30.552534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 551/1115] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:10:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:10:30.558567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d265a3-13b6-4286-b6b8-01cfb52a7d4f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:30.659884 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d265a3-13b6-4286-b6b8-01cfb52a7d4f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:30.662268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:30.693753 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460880f-d65e-43ba-9931-4c63af75d29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:30.694411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460880f-d65e-43ba-9931-4c63af75d29d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:30.694688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460880f-d65e-43ba-9931-4c63af75d29d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:30.694954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460880f-d65e-43ba-9931-4c63af75d29d None admin] PATH_INFO: `/v3/projects/9eb72c15001d4740beb34fac8e10d6b8` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:30.696326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460880f-d65e-43ba-9931-4c63af75d29d None admin] RBAC: Authorizing `identity:get_project(project_id=9eb72c15001d4740beb34fac8e10d6b8)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:30.697732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460880f-d65e-43ba-9931-4c63af75d29d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:30.699367 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 565/1116] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:10:30 2022] GET /identity/v3/projects/9eb72c15001d4740beb34fac8e10d6b8 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:30.706148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72478c-bc67-4395-aafd-e411384c40dc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:30.723916 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:30.806420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72478c-bc67-4395-aafd-e411384c40dc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:30.844466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714e996-fadc-401f-8275-b9bf6331d0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:30.845319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714e996-fadc-401f-8275-b9bf6331d0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:30.845741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714e996-fadc-401f-8275-b9bf6331d0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:30.846566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714e996-fadc-401f-8275-b9bf6331d0b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:30.847441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714e996-fadc-401f-8275-b9bf6331d0b0 None admin] RBAC: Adding query filter params (role.id=aa320f864d914f65bebe5463994ff536, scope.project.id=9eb72c15001d4740beb34fac8e10d6b8, user.id=1f59402ba955493e861c25a0ef67e0b7) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:30.848074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714e996-fadc-401f-8275-b9bf6331d0b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:30.849556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714e996-fadc-401f-8275-b9bf6331d0b0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:30.855338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 552/1117] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:10:30 2022] GET /identity/v3/role_assignments?user.id=1f59402ba955493e861c25a0ef67e0b7&scope.project.id=9eb72c15001d4740beb34fac8e10d6b8&role.id=aa320f864d914f65bebe5463994ff536 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:32.966788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e1c40-c2a4-4a68-8f18-910c965aceed None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:32.967363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e1c40-c2a4-4a68-8f18-910c965aceed None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:32.967363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e1c40-c2a4-4a68-8f18-910c965aceed None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:32.968645 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 566/1118] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:32.973694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e718a8-d3c5-43b9-8c8c-228f48c1aeda None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:32.974078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e718a8-d3c5-43b9-8c8c-228f48c1aeda None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:32.974271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e718a8-d3c5-43b9-8c8c-228f48c1aeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:32.992451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a0e718a8-d3c5-43b9-8c8c-228f48c1aeda None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:32.993625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a0e718a8-d3c5-43b9-8c8c-228f48c1aeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:33.058635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 553/1119] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:33.328019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e1c40-c2a4-4a68-8f18-910c965aceed None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:33.425940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e1c40-c2a4-4a68-8f18-910c965aceed None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:33.461213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f8f9c-303f-40ab-8fdf-708c677f7bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:33.461877 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f8f9c-303f-40ab-8fdf-708c677f7bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:33.462184 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f8f9c-303f-40ab-8fdf-708c677f7bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:33.462463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7f8f9c-303f-40ab-8fdf-708c677f7bd6 None admin] PATH_INFO: `/v3/roles/aa320f864d914f65bebe5463994ff536` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:33.463868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f8f9c-303f-40ab-8fdf-708c677f7bd6 None admin] RBAC: Authorizing `identity:get_role(role_id=aa320f864d914f65bebe5463994ff536)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:33.465222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7f8f9c-303f-40ab-8fdf-708c677f7bd6 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:33.466578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 567/1120] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:10:33 2022] GET /identity/v3/roles/aa320f864d914f65bebe5463994ff536 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:10:33.472848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e718a8-d3c5-43b9-8c8c-228f48c1aeda None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:33.484159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:33.571541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e718a8-d3c5-43b9-8c8c-228f48c1aeda None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:33.604480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfdc65-bd12-47c2-92a8-6b98dafe5ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:33.605035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfdc65-bd12-47c2-92a8-6b98dafe5ffd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:33.605315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfdc65-bd12-47c2-92a8-6b98dafe5ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:33.605588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfdc65-bd12-47c2-92a8-6b98dafe5ffd None admin] PATH_INFO: `/v3/users/1f59402ba955493e861c25a0ef67e0b7` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:33.606682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfdc65-bd12-47c2-92a8-6b98dafe5ffd None admin] RBAC: Authorizing `identity:get_user(user_id=1f59402ba955493e861c25a0ef67e0b7)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:33.608097 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfdc65-bd12-47c2-92a8-6b98dafe5ffd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:33.609673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 554/1121] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:10:33 2022] GET /identity/v3/users/1f59402ba955493e861c25a0ef67e0b7 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:33.616136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f8f9c-303f-40ab-8fdf-708c677f7bd6 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:33.656291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:33.713419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7f8f9c-303f-40ab-8fdf-708c677f7bd6 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:33.746420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c9632-4d89-4419-8c6a-9aede6f0c999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:33.747037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c9632-4d89-4419-8c6a-9aede6f0c999 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:33.747311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c9632-4d89-4419-8c6a-9aede6f0c999 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:33.747578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c9632-4d89-4419-8c6a-9aede6f0c999 None admin] PATH_INFO: `/v3/projects/9eb72c15001d4740beb34fac8e10d6b8` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:33.748664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c9632-4d89-4419-8c6a-9aede6f0c999 None admin] RBAC: Authorizing `identity:get_project(project_id=9eb72c15001d4740beb34fac8e10d6b8)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:33.750085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c9632-4d89-4419-8c6a-9aede6f0c999 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:33.751674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 568/1122] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:10:33 2022] GET /identity/v3/projects/9eb72c15001d4740beb34fac8e10d6b8 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:33.758011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfdc65-bd12-47c2-92a8-6b98dafe5ffd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:33.834798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:33.856311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfdc65-bd12-47c2-92a8-6b98dafe5ffd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:33.889134 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a812469b-d4f3-442b-80ba-a7bf65dccf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:33.889783 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812469b-d4f3-442b-80ba-a7bf65dccf97 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:33.890083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812469b-d4f3-442b-80ba-a7bf65dccf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:33.890363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812469b-d4f3-442b-80ba-a7bf65dccf97 None admin] PATH_INFO: `/v3/projects/9eb72c15001d4740beb34fac8e10d6b8/users/1f59402ba955493e861c25a0ef67e0b7/roles/aa320f864d914f65bebe5463994ff536` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:33.893024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812469b-d4f3-442b-80ba-a7bf65dccf97 None admin] RBAC: Authorizing `identity:create_grant(project_id=9eb72c15001d4740beb34fac8e10d6b8, user_id=1f59402ba955493e861c25a0ef67e0b7, role_id=aa320f864d914f65bebe5463994ff536)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:33.894494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a812469b-d4f3-442b-80ba-a7bf65dccf97 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:33.906871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 555/1123] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:10:33 2022] PUT /identity/v3/projects/9eb72c15001d4740beb34fac8e10d6b8/users/1f59402ba955493e861c25a0ef67e0b7/roles/aa320f864d914f65bebe5463994ff536 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:10:36.068696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3de56-61ae-4064-900d-a7fb367bb7c9 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:36.069263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3de56-61ae-4064-900d-a7fb367bb7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:36.069263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3de56-61ae-4064-900d-a7fb367bb7c9 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:36.070911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 569/1124] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:36.076498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0910a-4e27-4e40-9765-35e14a67c9e4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:36.077008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0910a-4e27-4e40-9765-35e14a67c9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:36.077121 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0910a-4e27-4e40-9765-35e14a67c9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:36.094948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-54d0910a-4e27-4e40-9765-35e14a67c9e4 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:36.096193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-54d0910a-4e27-4e40-9765-35e14a67c9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:36.160722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 556/1125] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:10:36.436538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3de56-61ae-4064-900d-a7fb367bb7c9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:36.534736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3de56-61ae-4064-900d-a7fb367bb7c9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:36.543562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:36.567940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ab055-f4bb-4a80-9120-9ad008078276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:36.568556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ab055-f4bb-4a80-9120-9ad008078276 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:36.568831 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ab055-f4bb-4a80-9120-9ad008078276 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:36.569097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4ab055-f4bb-4a80-9120-9ad008078276 None admin] PATH_INFO: `/v3/roles/aa320f864d914f65bebe5463994ff536` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:36.570736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ab055-f4bb-4a80-9120-9ad008078276 None admin] RBAC: Authorizing `identity:get_role(role_id=aa320f864d914f65bebe5463994ff536)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:36.572074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4ab055-f4bb-4a80-9120-9ad008078276 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:36.573383 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 570/1126] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:10:36 2022] GET /identity/v3/roles/aa320f864d914f65bebe5463994ff536 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:36.579970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0910a-4e27-4e40-9765-35e14a67c9e4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:36.644625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:36.678390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0910a-4e27-4e40-9765-35e14a67c9e4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:36.711310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574f7b2-ca3e-41ef-8855-1614675fda45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:36.711864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574f7b2-ca3e-41ef-8855-1614675fda45 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:36.712145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574f7b2-ca3e-41ef-8855-1614675fda45 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:36.712420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574f7b2-ca3e-41ef-8855-1614675fda45 None admin] PATH_INFO: `/v3/users/1f59402ba955493e861c25a0ef67e0b7` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:36.713547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574f7b2-ca3e-41ef-8855-1614675fda45 None admin] RBAC: Authorizing `identity:get_user(user_id=1f59402ba955493e861c25a0ef67e0b7)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:36.714987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574f7b2-ca3e-41ef-8855-1614675fda45 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:36.716551 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 557/1127] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:10:36 2022] GET /identity/v3/users/1f59402ba955493e861c25a0ef67e0b7 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:36.723093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ab055-f4bb-4a80-9120-9ad008078276 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:36.818504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4ab055-f4bb-4a80-9120-9ad008078276 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:36.852235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8887ac5-68ed-4cf3-87f4-c0ae8e3b4aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:36.852829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8887ac5-68ed-4cf3-87f4-c0ae8e3b4aef None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:36.853102 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8887ac5-68ed-4cf3-87f4-c0ae8e3b4aef None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:36.853366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8887ac5-68ed-4cf3-87f4-c0ae8e3b4aef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:36.896445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:36.919058 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8887ac5-68ed-4cf3-87f4-c0ae8e3b4aef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:36.920517 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8887ac5-68ed-4cf3-87f4-c0ae8e3b4aef None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:37.015248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 571/1128] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:10:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 21 16:10:37.020101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574f7b2-ca3e-41ef-8855-1614675fda45 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:37.116936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574f7b2-ca3e-41ef-8855-1614675fda45 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:37.155311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20323c83-340f-4f9e-9c7d-faf2e07a060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:37.155945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20323c83-340f-4f9e-9c7d-faf2e07a060e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:37.156225 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20323c83-340f-4f9e-9c7d-faf2e07a060e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:37.156497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20323c83-340f-4f9e-9c7d-faf2e07a060e None admin] PATH_INFO: `/v3/projects/9eb72c15001d4740beb34fac8e10d6b8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:37.157620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20323c83-340f-4f9e-9c7d-faf2e07a060e None admin] RBAC: Authorizing `identity:get_project(project_id=9eb72c15001d4740beb34fac8e10d6b8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:37.159102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20323c83-340f-4f9e-9c7d-faf2e07a060e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:37.160736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 558/1129] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:10:37 2022] GET /identity/v3/projects/9eb72c15001d4740beb34fac8e10d6b8 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:37.166965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8887ac5-68ed-4cf3-87f4-c0ae8e3b4aef None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:37.242069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:37.266508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8887ac5-68ed-4cf3-87f4-c0ae8e3b4aef None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:37.300812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5418a-ae87-489d-a327-74bc5a965a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:37.301416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5418a-ae87-489d-a327-74bc5a965a4b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:37.301697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5418a-ae87-489d-a327-74bc5a965a4b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:37.301971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5418a-ae87-489d-a327-74bc5a965a4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:37.302732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5418a-ae87-489d-a327-74bc5a965a4b None admin] RBAC: Adding query filter params (role.id=aa320f864d914f65bebe5463994ff536, scope.project.id=9eb72c15001d4740beb34fac8e10d6b8, user.id=1f59402ba955493e861c25a0ef67e0b7) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:37.303044 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5418a-ae87-489d-a327-74bc5a965a4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:37.304460 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5418a-ae87-489d-a327-74bc5a965a4b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:37.310321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 572/1130] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:10:37 2022] GET /identity/v3/role_assignments?user.id=1f59402ba955493e861c25a0ef67e0b7&scope.project.id=9eb72c15001d4740beb34fac8e10d6b8&role.id=aa320f864d914f65bebe5463994ff536 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:39.514873 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a8ec0-e846-4cfd-9418-bf19114b3e18 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:39.515414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a8ec0-e846-4cfd-9418-bf19114b3e18 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:39.515414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a8ec0-e846-4cfd-9418-bf19114b3e18 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:39.516725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 559/1131] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:39.522639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba5f24-467b-44a4-bf18-59bd313026c7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:39.523075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba5f24-467b-44a4-bf18-59bd313026c7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:39.523175 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba5f24-467b-44a4-bf18-59bd313026c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:39.541454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d5ba5f24-467b-44a4-bf18-59bd313026c7 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:39.542660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d5ba5f24-467b-44a4-bf18-59bd313026c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:39.606886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 573/1132] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:39.873088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a8ec0-e846-4cfd-9418-bf19114b3e18 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:39.876889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:39.973040 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a8ec0-e846-4cfd-9418-bf19114b3e18 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:40.006787 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132904be-7363-496b-a447-2eab1e5c405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:40.007404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132904be-7363-496b-a447-2eab1e5c405e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:40.007682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132904be-7363-496b-a447-2eab1e5c405e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:40.007951 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132904be-7363-496b-a447-2eab1e5c405e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:40.009043 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132904be-7363-496b-a447-2eab1e5c405e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:40.010475 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132904be-7363-496b-a447-2eab1e5c405e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:40.012016 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 560/1133] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:40.018441 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba5f24-467b-44a4-bf18-59bd313026c7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:40.085021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:40.116815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba5f24-467b-44a4-bf18-59bd313026c7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:40.150091 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c7966-6b65-4a90-b919-52fad8587dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:40.150626 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c7966-6b65-4a90-b919-52fad8587dfb None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:40.150900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c7966-6b65-4a90-b919-52fad8587dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:40.151168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c7966-6b65-4a90-b919-52fad8587dfb None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:40.151818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c7966-6b65-4a90-b919-52fad8587dfb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:40.153222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c7966-6b65-4a90-b919-52fad8587dfb None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:40.178295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 574/1134] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:10:40 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:10:42.378102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7336-d71b-4942-9ccd-868bf83eaae5 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:42.378661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7336-d71b-4942-9ccd-868bf83eaae5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:42.378661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad7336-d71b-4942-9ccd-868bf83eaae5 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:42.380034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 561/1135] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:42.385210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1f960-104b-4e07-bd2b-5bafb9ced5d5 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:42.385656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1f960-104b-4e07-bd2b-5bafb9ced5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:42.385802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1f960-104b-4e07-bd2b-5bafb9ced5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:42.403825 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e6c1f960-104b-4e07-bd2b-5bafb9ced5d5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:42.405004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e6c1f960-104b-4e07-bd2b-5bafb9ced5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:42.471530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 575/1136] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:42.761747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7336-d71b-4942-9ccd-868bf83eaae5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:42.828957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:42.858548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad7336-d71b-4942-9ccd-868bf83eaae5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:42.890831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950009d-4b83-48fc-975c-68692bfba66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:42.891450 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950009d-4b83-48fc-975c-68692bfba66c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:42.892241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950009d-4b83-48fc-975c-68692bfba66c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:42.892511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950009d-4b83-48fc-975c-68692bfba66c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:42.893630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950009d-4b83-48fc-975c-68692bfba66c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:42.895082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950009d-4b83-48fc-975c-68692bfba66c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:42.896628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 562/1137] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:10:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:42.902394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1f960-104b-4e07-bd2b-5bafb9ced5d5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:42.953028 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:43.000029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1f960-104b-4e07-bd2b-5bafb9ced5d5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:43.033872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f540aeb-09c6-4836-9b75-cb4502d84353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:43.034432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f540aeb-09c6-4836-9b75-cb4502d84353 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:43.034709 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f540aeb-09c6-4836-9b75-cb4502d84353 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:43.034979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f540aeb-09c6-4836-9b75-cb4502d84353 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:43.035628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f540aeb-09c6-4836-9b75-cb4502d84353 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:43.037032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f540aeb-09c6-4836-9b75-cb4502d84353 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:43.058000 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 576/1138] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:10:42 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:10:45.268105 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11cebd-af98-4b94-886d-a5418e3aa8aa None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:45.268632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11cebd-af98-4b94-886d-a5418e3aa8aa None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:45.268632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11cebd-af98-4b94-886d-a5418e3aa8aa None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:45.269980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 563/1139] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:45.275496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e64c0-fccc-4d5a-95ca-647ff319a812 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:45.275905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e64c0-fccc-4d5a-95ca-647ff319a812 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:45.276036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e64c0-fccc-4d5a-95ca-647ff319a812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:45.295181 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-cf3e64c0-fccc-4d5a-95ca-647ff319a812 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:45.296834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cf3e64c0-fccc-4d5a-95ca-647ff319a812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:45.367660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 577/1140] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:45.637295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11cebd-af98-4b94-886d-a5418e3aa8aa None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:45.736060 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11cebd-af98-4b94-886d-a5418e3aa8aa None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:45.768822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0484b2e-c500-40b3-8332-44e68cf5a4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:45.769436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0484b2e-c500-40b3-8332-44e68cf5a4ad None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:45.769718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0484b2e-c500-40b3-8332-44e68cf5a4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:45.769991 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0484b2e-c500-40b3-8332-44e68cf5a4ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:45.777387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:45.780658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0484b2e-c500-40b3-8332-44e68cf5a4ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d0484b2e-c500-40b3-8332-44e68cf5a4ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:10:45.785664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:45.786883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:45.787966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 564/1141] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:10:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:45.792806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e64c0-fccc-4d5a-95ca-647ff319a812 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:45.831807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:45.889894 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e64c0-fccc-4d5a-95ca-647ff319a812 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:45.921986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebd5f5-d15e-4bb2-9baa-84094738e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:45.922623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd5f5-d15e-4bb2-9baa-84094738e44e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:45.923505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd5f5-d15e-4bb2-9baa-84094738e44e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:45.923773 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd5f5-d15e-4bb2-9baa-84094738e44e None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:45.924425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebd5f5-d15e-4bb2-9baa-84094738e44e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:45.924725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebd5f5-d15e-4bb2-9baa-84094738e44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:45.926088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebd5f5-d15e-4bb2-9baa-84094738e44e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:45.933899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 578/1142] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:10:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:45.940669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0484b2e-c500-40b3-8332-44e68cf5a4ad None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:46.039273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0484b2e-c500-40b3-8332-44e68cf5a4ad None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:46.073524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ef1b0-32e7-4e22-9cda-0d03d948d716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:46.074180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ef1b0-32e7-4e22-9cda-0d03d948d716 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:46.074464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ef1b0-32e7-4e22-9cda-0d03d948d716 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:46.074739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ef1b0-32e7-4e22-9cda-0d03d948d716 None admin] PATH_INFO: `/v3/users/349438d2b780436aa4885bf07048cfe1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:46.087661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ef1b0-32e7-4e22-9cda-0d03d948d716 None admin] RBAC: Authorizing `identity:get_user(user_id=349438d2b780436aa4885bf07048cfe1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:46.089090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ef1b0-32e7-4e22-9cda-0d03d948d716 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:46.090731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 565/1143] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:10:45 2022] GET /identity/v3/users/349438d2b780436aa4885bf07048cfe1 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:46.097263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebd5f5-d15e-4bb2-9baa-84094738e44e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:46.191994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:46.195232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebd5f5-d15e-4bb2-9baa-84094738e44e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:46.228325 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f7f9a-a281-4693-9944-1ac39edb7d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:46.228925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f7f9a-a281-4693-9944-1ac39edb7d60 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:46.229201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f7f9a-a281-4693-9944-1ac39edb7d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:46.229471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f7f9a-a281-4693-9944-1ac39edb7d60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:46.294907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f7f9a-a281-4693-9944-1ac39edb7d60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:46.296345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f7f9a-a281-4693-9944-1ac39edb7d60 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:46.393743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:46.397562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 579/1144] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:10:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:10:46.403812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ef1b0-32e7-4e22-9cda-0d03d948d716 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:46.468839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:46.515170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ef1b0-32e7-4e22-9cda-0d03d948d716 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:46.549004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888894fe-fe8c-4aeb-b0ef-76a9d34adc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:46.549625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888894fe-fe8c-4aeb-b0ef-76a9d34adc11 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:46.549901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888894fe-fe8c-4aeb-b0ef-76a9d34adc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:46.550205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888894fe-fe8c-4aeb-b0ef-76a9d34adc11 None admin] PATH_INFO: `/v3/projects/3a002da537e045fb8b8ea4ae2be40b76` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:46.551302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888894fe-fe8c-4aeb-b0ef-76a9d34adc11 None admin] RBAC: Authorizing `identity:get_project(project_id=3a002da537e045fb8b8ea4ae2be40b76)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:46.552745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888894fe-fe8c-4aeb-b0ef-76a9d34adc11 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:46.554433 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 566/1145] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:10:46 2022] GET /identity/v3/projects/3a002da537e045fb8b8ea4ae2be40b76 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:46.560663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f7f9a-a281-4693-9944-1ac39edb7d60 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:46.661092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f7f9a-a281-4693-9944-1ac39edb7d60 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:46.695003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ae22d-3098-48b8-b48b-9b34d674d7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:46.695602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ae22d-3098-48b8-b48b-9b34d674d7bd None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:46.695880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ae22d-3098-48b8-b48b-9b34d674d7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:46.696153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ae22d-3098-48b8-b48b-9b34d674d7bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:46.696878 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ae22d-3098-48b8-b48b-9b34d674d7bd None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=3a002da537e045fb8b8ea4ae2be40b76, user.id=349438d2b780436aa4885bf07048cfe1) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:46.697188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ae22d-3098-48b8-b48b-9b34d674d7bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:46.698632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ae22d-3098-48b8-b48b-9b34d674d7bd None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:46.703848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 580/1146] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:10:46 2022] GET /identity/v3/role_assignments?user.id=349438d2b780436aa4885bf07048cfe1&scope.project.id=3a002da537e045fb8b8ea4ae2be40b76&role.id=459d01a218314543a7a8329e88b78331 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:48.846380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec6ff8-cbf2-4f4b-a85d-f7c1c6962c7c None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:48.846902 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec6ff8-cbf2-4f4b-a85d-f7c1c6962c7c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:48.846902 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec6ff8-cbf2-4f4b-a85d-f7c1c6962c7c None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:48.848158 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 567/1147] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:48.853363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1cbe2-7dda-4cd1-9909-2b1527be184f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:48.853764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1cbe2-7dda-4cd1-9909-2b1527be184f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:48.853913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1cbe2-7dda-4cd1-9909-2b1527be184f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:48.876917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4cd1cbe2-7dda-4cd1-9909-2b1527be184f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:48.878790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4cd1cbe2-7dda-4cd1-9909-2b1527be184f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:48.935047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:48.944967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 581/1148] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:49.211041 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec6ff8-cbf2-4f4b-a85d-f7c1c6962c7c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:49.308891 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec6ff8-cbf2-4f4b-a85d-f7c1c6962c7c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:49.333812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:49.342077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f867e4-4b1f-4220-b054-420a60fb3c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:49.342689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f867e4-4b1f-4220-b054-420a60fb3c47 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:49.342963 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f867e4-4b1f-4220-b054-420a60fb3c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:49.343233 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f867e4-4b1f-4220-b054-420a60fb3c47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:49.353391 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f867e4-4b1f-4220-b054-420a60fb3c47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-58f867e4-4b1f-4220-b054-420a60fb3c47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:10:49.358118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:49.359197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:49.361160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 568/1149] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:10:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:49.365935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1cbe2-7dda-4cd1-9909-2b1527be184f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:49.462485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1cbe2-7dda-4cd1-9909-2b1527be184f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:49.495644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb54fb-751d-4f83-b62d-be698298a42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:49.496250 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb54fb-751d-4f83-b62d-be698298a42c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:49.496530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb54fb-751d-4f83-b62d-be698298a42c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:49.496802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb54fb-751d-4f83-b62d-be698298a42c None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:49.497468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb54fb-751d-4f83-b62d-be698298a42c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:49.497772 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb54fb-751d-4f83-b62d-be698298a42c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:49.499152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb54fb-751d-4f83-b62d-be698298a42c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:49.507186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 582/1150] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:10:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:49.513021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f867e4-4b1f-4220-b054-420a60fb3c47 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:49.614057 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f867e4-4b1f-4220-b054-420a60fb3c47 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:49.650321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f49070-3236-49d4-b1ad-5ebc78698479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:49.650940 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f49070-3236-49d4-b1ad-5ebc78698479 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:49.651213 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f49070-3236-49d4-b1ad-5ebc78698479 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:49.651479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f49070-3236-49d4-b1ad-5ebc78698479 None admin] PATH_INFO: `/v3/users/349438d2b780436aa4885bf07048cfe1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:49.652580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f49070-3236-49d4-b1ad-5ebc78698479 None admin] RBAC: Authorizing `identity:get_user(user_id=349438d2b780436aa4885bf07048cfe1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:49.653986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f49070-3236-49d4-b1ad-5ebc78698479 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:49.655577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 569/1151] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:10:49 2022] GET /identity/v3/users/349438d2b780436aa4885bf07048cfe1 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:49.662139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb54fb-751d-4f83-b62d-be698298a42c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:49.716632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:49.760299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb54fb-751d-4f83-b62d-be698298a42c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:49.794565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6039f79b-d804-47fc-9ac2-c255eaaeab76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:49.795219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039f79b-d804-47fc-9ac2-c255eaaeab76 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:49.795501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039f79b-d804-47fc-9ac2-c255eaaeab76 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:49.795789 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039f79b-d804-47fc-9ac2-c255eaaeab76 None admin] PATH_INFO: `/v3/projects/3a002da537e045fb8b8ea4ae2be40b76` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:49.796908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039f79b-d804-47fc-9ac2-c255eaaeab76 None admin] RBAC: Authorizing `identity:get_project(project_id=3a002da537e045fb8b8ea4ae2be40b76)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:49.798365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6039f79b-d804-47fc-9ac2-c255eaaeab76 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:49.800034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 583/1152] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:10:49 2022] GET /identity/v3/projects/3a002da537e045fb8b8ea4ae2be40b76 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:49.806160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f49070-3236-49d4-b1ad-5ebc78698479 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:49.860224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:49.908434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f49070-3236-49d4-b1ad-5ebc78698479 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:49.941900 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c14075-8c7e-4270-805d-fb913f1a95f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:49.942580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c14075-8c7e-4270-805d-fb913f1a95f4 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:49.942860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c14075-8c7e-4270-805d-fb913f1a95f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:49.943133 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c14075-8c7e-4270-805d-fb913f1a95f4 None admin] PATH_INFO: `/v3/projects/3a002da537e045fb8b8ea4ae2be40b76/users/349438d2b780436aa4885bf07048cfe1/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:49.945263 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c14075-8c7e-4270-805d-fb913f1a95f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a002da537e045fb8b8ea4ae2be40b76, user_id=349438d2b780436aa4885bf07048cfe1, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:49.946724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c14075-8c7e-4270-805d-fb913f1a95f4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:49.959935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 570/1153] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:10:49 2022] PUT /identity/v3/projects/3a002da537e045fb8b8ea4ae2be40b76/users/349438d2b780436aa4885bf07048cfe1/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:10:52.174883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b94ad-6e18-4c43-bde9-c02d4cfabcec None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:52.175522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b94ad-6e18-4c43-bde9-c02d4cfabcec None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:52.175522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b94ad-6e18-4c43-bde9-c02d4cfabcec None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:52.176758 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 584/1154] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:52.182002 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9f87a-a792-481d-bd4a-3292e234bf17 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:52.182432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9f87a-a792-481d-bd4a-3292e234bf17 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:52.182570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9f87a-a792-481d-bd4a-3292e234bf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:52.201017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-41e9f87a-a792-481d-bd4a-3292e234bf17 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:52.202230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-41e9f87a-a792-481d-bd4a-3292e234bf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:52.266871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 571/1155] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:52.537009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b94ad-6e18-4c43-bde9-c02d4cfabcec None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:52.636643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b94ad-6e18-4c43-bde9-c02d4cfabcec None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:52.658821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:52.669664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1a2d3-84b8-49b7-9ab9-d78975a0f2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:52.670328 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1a2d3-84b8-49b7-9ab9-d78975a0f2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:52.670607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1a2d3-84b8-49b7-9ab9-d78975a0f2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:52.670879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1a2d3-84b8-49b7-9ab9-d78975a0f2f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:52.682092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1a2d3-84b8-49b7-9ab9-d78975a0f2f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-98f1a2d3-84b8-49b7-9ab9-d78975a0f2f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:10:52.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:52.688579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:10:52.689711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 585/1156] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:10:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:52.694876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e9f87a-a792-481d-bd4a-3292e234bf17 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:52.739484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:52.793433 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e9f87a-a792-481d-bd4a-3292e234bf17 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:52.826683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17c57a-ab17-4b59-b8b2-b3c31f02023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:52.827321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c57a-ab17-4b59-b8b2-b3c31f02023a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:52.827617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c57a-ab17-4b59-b8b2-b3c31f02023a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:52.827892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17c57a-ab17-4b59-b8b2-b3c31f02023a None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:52.829046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17c57a-ab17-4b59-b8b2-b3c31f02023a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:52.829349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17c57a-ab17-4b59-b8b2-b3c31f02023a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:52.830791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17c57a-ab17-4b59-b8b2-b3c31f02023a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:52.838708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 572/1157] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:10:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:52.845431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1a2d3-84b8-49b7-9ab9-d78975a0f2f5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:52.942950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1a2d3-84b8-49b7-9ab9-d78975a0f2f5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:52.976233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9a80f-5aeb-419a-b14a-d6c2ce933c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:52.976856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9a80f-5aeb-419a-b14a-d6c2ce933c59 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:52.977135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9a80f-5aeb-419a-b14a-d6c2ce933c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:52.977408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e9a80f-5aeb-419a-b14a-d6c2ce933c59 None admin] PATH_INFO: `/v3/users/349438d2b780436aa4885bf07048cfe1` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:52.978546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9a80f-5aeb-419a-b14a-d6c2ce933c59 None admin] RBAC: Authorizing `identity:get_user(user_id=349438d2b780436aa4885bf07048cfe1)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:52.979962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e9a80f-5aeb-419a-b14a-d6c2ce933c59 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:52.981517 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 586/1158] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:10:52 2022] GET /identity/v3/users/349438d2b780436aa4885bf07048cfe1 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:52.987563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17c57a-ab17-4b59-b8b2-b3c31f02023a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:53.085711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17c57a-ab17-4b59-b8b2-b3c31f02023a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:53.092549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:53.120166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd0ba4-81d8-444b-ad8c-827ac576ba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:53.120761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd0ba4-81d8-444b-ad8c-827ac576ba00 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:53.121041 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd0ba4-81d8-444b-ad8c-827ac576ba00 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:53.121314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcd0ba4-81d8-444b-ad8c-827ac576ba00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:53.190649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd0ba4-81d8-444b-ad8c-827ac576ba00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:53.192124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcd0ba4-81d8-444b-ad8c-827ac576ba00 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:53.292031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 573/1159] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:10:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:10:53.298852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9a80f-5aeb-419a-b14a-d6c2ce933c59 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:53.352870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:53.412979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e9a80f-5aeb-419a-b14a-d6c2ce933c59 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:53.448769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a4491-62e1-462e-ac1e-30d9d0c65ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:53.449394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a4491-62e1-462e-ac1e-30d9d0c65ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:53.449674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a4491-62e1-462e-ac1e-30d9d0c65ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:53.449947 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a4491-62e1-462e-ac1e-30d9d0c65ae7 None admin] PATH_INFO: `/v3/projects/3a002da537e045fb8b8ea4ae2be40b76` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:53.451090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a4491-62e1-462e-ac1e-30d9d0c65ae7 None admin] RBAC: Authorizing `identity:get_project(project_id=3a002da537e045fb8b8ea4ae2be40b76)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:53.452507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a4491-62e1-462e-ac1e-30d9d0c65ae7 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:53.454262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 587/1160] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:10:53 2022] GET /identity/v3/projects/3a002da537e045fb8b8ea4ae2be40b76 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:53.460467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd0ba4-81d8-444b-ad8c-827ac576ba00 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:53.468524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:53.558336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcd0ba4-81d8-444b-ad8c-827ac576ba00 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:53.591248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5a9a7-5281-4cf2-ad04-29a17abf8dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:53.591830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a9a7-5281-4cf2-ad04-29a17abf8dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:53.592107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a9a7-5281-4cf2-ad04-29a17abf8dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:53.592378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5a9a7-5281-4cf2-ad04-29a17abf8dc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:53.593086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5a9a7-5281-4cf2-ad04-29a17abf8dc3 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=3a002da537e045fb8b8ea4ae2be40b76, user.id=349438d2b780436aa4885bf07048cfe1) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:53.593390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5a9a7-5281-4cf2-ad04-29a17abf8dc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:53.595351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5a9a7-5281-4cf2-ad04-29a17abf8dc3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:53.600921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 574/1161] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:10:53 2022] GET /identity/v3/role_assignments?user.id=349438d2b780436aa4885bf07048cfe1&scope.project.id=3a002da537e045fb8b8ea4ae2be40b76&role.id=459d01a218314543a7a8329e88b78331 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:55.793466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de5c81-36ea-4fb5-9e29-005092bc19e9 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:55.793965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de5c81-36ea-4fb5-9e29-005092bc19e9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:55.793965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de5c81-36ea-4fb5-9e29-005092bc19e9 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:55.795303 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 588/1162] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:55.800575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d0d62-9db5-4d11-894e-18e89757be4f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:55.800982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d0d62-9db5-4d11-894e-18e89757be4f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:55.801124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d0d62-9db5-4d11-894e-18e89757be4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:55.818840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-057d0d62-9db5-4d11-894e-18e89757be4f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:55.820283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-057d0d62-9db5-4d11-894e-18e89757be4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:55.881988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:55.886942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 575/1163] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:56.158994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de5c81-36ea-4fb5-9e29-005092bc19e9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:56.257512 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de5c81-36ea-4fb5-9e29-005092bc19e9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:56.276449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:56.290597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d17fe-b6c5-4ff2-a66a-8d53c09bbe95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:56.291207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d17fe-b6c5-4ff2-a66a-8d53c09bbe95 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:56.291486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d17fe-b6c5-4ff2-a66a-8d53c09bbe95 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:56.291759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7d17fe-b6c5-4ff2-a66a-8d53c09bbe95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:56.361228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d17fe-b6c5-4ff2-a66a-8d53c09bbe95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:56.362694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7d17fe-b6c5-4ff2-a66a-8d53c09bbe95 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:56.459030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 589/1164] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:10:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:10:56.464931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d0d62-9db5-4d11-894e-18e89757be4f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:56.568627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d0d62-9db5-4d11-894e-18e89757be4f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:56.601824 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356b54d-e171-4c4b-9979-4d4ea51833ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:56.602475 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356b54d-e171-4c4b-9979-4d4ea51833ab None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:56.602755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356b54d-e171-4c4b-9979-4d4ea51833ab None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:56.603023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356b54d-e171-4c4b-9979-4d4ea51833ab None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:56.610287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356b54d-e171-4c4b-9979-4d4ea51833ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:56.611717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356b54d-e171-4c4b-9979-4d4ea51833ab None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1356b54d-e171-4c4b-9979-4d4ea51833ab None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:10:56.622966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:10:56.624219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:10:56.625470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 576/1165] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Sep 21 16:10:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:10:56.630291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d17fe-b6c5-4ff2-a66a-8d53c09bbe95 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:56.650799 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:56.728351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7d17fe-b6c5-4ff2-a66a-8d53c09bbe95 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:56.762061 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a1c02-38d9-41ce-a588-0f70f6b77379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:56.762710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a1c02-38d9-41ce-a588-0f70f6b77379 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:56.762991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a1c02-38d9-41ce-a588-0f70f6b77379 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:56.763264 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a1c02-38d9-41ce-a588-0f70f6b77379 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:56.763931 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a1c02-38d9-41ce-a588-0f70f6b77379 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:10:56.764235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a1c02-38d9-41ce-a588-0f70f6b77379 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:56.765674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158a1c02-38d9-41ce-a588-0f70f6b77379 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:56.774026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 590/1166] 10.222.0.64 () {56 vars in 1115 bytes} [Wed Sep 21 16:10:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:10:58.962018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661568ae-94ca-4e82-aea5-fae06589e9cf None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:58.962573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661568ae-94ca-4e82-aea5-fae06589e9cf None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:58.962573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661568ae-94ca-4e82-aea5-fae06589e9cf None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:58.963924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 577/1167] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:10:58.969568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ecaa5-b3e5-4c3c-8f1f-f433e481acee None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:58.969984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ecaa5-b3e5-4c3c-8f1f-f433e481acee None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:58.970165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ecaa5-b3e5-4c3c-8f1f-f433e481acee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:58.988505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-176ecaa5-b3e5-4c3c-8f1f-f433e481acee None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:10:58.989698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-176ecaa5-b3e5-4c3c-8f1f-f433e481acee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:10:59.057782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 591/1168] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:10:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:10:59.329218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661568ae-94ca-4e82-aea5-fae06589e9cf None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:10:59.388921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:10:59.432156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661568ae-94ca-4e82-aea5-fae06589e9cf None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:10:59.465565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a6259b-ad1d-4a70-9706-2ebcd86edcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:10:59.466191 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6259b-ad1d-4a70-9706-2ebcd86edcdd None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:10:59.466474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6259b-ad1d-4a70-9706-2ebcd86edcdd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:10:59.466744 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a6259b-ad1d-4a70-9706-2ebcd86edcdd None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:10:59.467410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6259b-ad1d-4a70-9706-2ebcd86edcdd None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:10:59.469319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a6259b-ad1d-4a70-9706-2ebcd86edcdd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:10:59.486930 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 578/1169] 10.222.0.64 () {60 vars in 1138 bytes} [Wed Sep 21 16:10:59 2022] POST /identity/v3/domains => generated 267 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:11:01.670520 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73711ef3-981c-4959-80e8-44d99725d32d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:01.671075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73711ef3-981c-4959-80e8-44d99725d32d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:01.671187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73711ef3-981c-4959-80e8-44d99725d32d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:01.673104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 592/1170] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:11:01.678448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd44689-51ef-450a-b7dd-4a91cb3c7027 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:01.678870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd44689-51ef-450a-b7dd-4a91cb3c7027 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:01.678987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd44689-51ef-450a-b7dd-4a91cb3c7027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:01.697544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4fd44689-51ef-450a-b7dd-4a91cb3c7027 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:11:01.698756 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4fd44689-51ef-450a-b7dd-4a91cb3c7027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:11:01.762897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 579/1171] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:11:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:11:02.028779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73711ef3-981c-4959-80e8-44d99725d32d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:02.046313 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:02.139712 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73711ef3-981c-4959-80e8-44d99725d32d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:02.176318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c3b4b-8bed-41ec-9a99-429f3beb3052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:02.177060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c3b4b-8bed-41ec-9a99-429f3beb3052 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:02.177376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c3b4b-8bed-41ec-9a99-429f3beb3052 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:02.177676 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c3b4b-8bed-41ec-9a99-429f3beb3052 None admin] PATH_INFO: `/v3/domains/64ffbdace3be45878c7cefa670518dbf` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:02.188480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c3b4b-8bed-41ec-9a99-429f3beb3052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=64ffbdace3be45878c7cefa670518dbf)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:02.189893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c3b4b-8bed-41ec-9a99-429f3beb3052 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:02.191519 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 593/1172] 10.222.0.64 () {56 vars in 1183 bytes} [Wed Sep 21 16:11:02 2022] GET /identity/v3/domains/64ffbdace3be45878c7cefa670518dbf => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:02.197843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd44689-51ef-450a-b7dd-4a91cb3c7027 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:02.246576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:02.296170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd44689-51ef-450a-b7dd-4a91cb3c7027 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:02.329560 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b491ce6-5e0e-41d7-a7b5-44828797fa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:02.330195 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b491ce6-5e0e-41d7-a7b5-44828797fa5c None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:02.330474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b491ce6-5e0e-41d7-a7b5-44828797fa5c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:02.330747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b491ce6-5e0e-41d7-a7b5-44828797fa5c None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:02.331413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b491ce6-5e0e-41d7-a7b5-44828797fa5c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:02.332836 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b491ce6-5e0e-41d7-a7b5-44828797fa5c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:02.364183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 580/1173] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:11:02 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:11:04.562241 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2353d-c25f-4dc6-b53f-c37d5a821010 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:04.562795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2353d-c25f-4dc6-b53f-c37d5a821010 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:04.563210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2353d-c25f-4dc6-b53f-c37d5a821010 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:04.565458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 594/1174] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:11:04.571118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7ad26-19a3-49d6-ac28-cf2fc21d77c1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:04.571572 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7ad26-19a3-49d6-ac28-cf2fc21d77c1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:04.571667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7ad26-19a3-49d6-ac28-cf2fc21d77c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:04.590584 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-05c7ad26-19a3-49d6-ac28-cf2fc21d77c1 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:11:04.591775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-05c7ad26-19a3-49d6-ac28-cf2fc21d77c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:11:04.657249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 581/1175] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:11:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:11:04.921060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2353d-c25f-4dc6-b53f-c37d5a821010 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:04.995898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:05.027133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2353d-c25f-4dc6-b53f-c37d5a821010 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:05.059169 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f553b2-0767-4573-81db-48e901a8830f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:05.059788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f553b2-0767-4573-81db-48e901a8830f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:05.060061 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f553b2-0767-4573-81db-48e901a8830f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:05.060328 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f553b2-0767-4573-81db-48e901a8830f None admin] PATH_INFO: `/v3/domains/64ffbdace3be45878c7cefa670518dbf` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:05.061413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f553b2-0767-4573-81db-48e901a8830f None admin] RBAC: Authorizing `identity:get_domain(domain_id=64ffbdace3be45878c7cefa670518dbf)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:05.063398 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f553b2-0767-4573-81db-48e901a8830f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:05.064940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 595/1176] 10.222.0.64 () {56 vars in 1183 bytes} [Wed Sep 21 16:11:04 2022] GET /identity/v3/domains/64ffbdace3be45878c7cefa670518dbf => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:05.071220 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7ad26-19a3-49d6-ac28-cf2fc21d77c1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:05.101436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:05.169556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7ad26-19a3-49d6-ac28-cf2fc21d77c1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:05.203585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6402a-55ea-4339-8c8b-988472ed2d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:05.204105 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6402a-55ea-4339-8c8b-988472ed2d94 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:05.204385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6402a-55ea-4339-8c8b-988472ed2d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:05.204658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6402a-55ea-4339-8c8b-988472ed2d94 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:05.205312 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6402a-55ea-4339-8c8b-988472ed2d94 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:05.206762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6402a-55ea-4339-8c8b-988472ed2d94 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:05.227236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 582/1177] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:11:05 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:11:07.459628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfa3de-c1bf-4a79-b5bf-53d90f293d6b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:07.460185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfa3de-c1bf-4a79-b5bf-53d90f293d6b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:07.460185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bfa3de-c1bf-4a79-b5bf-53d90f293d6b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:07.461543 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 596/1178] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:11:07.466943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64766ba8-53cc-4b26-bf5c-99c7fc49b5f4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:07.467351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64766ba8-53cc-4b26-bf5c-99c7fc49b5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:07.467487 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64766ba8-53cc-4b26-bf5c-99c7fc49b5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:07.486056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-64766ba8-53cc-4b26-bf5c-99c7fc49b5f4 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:11:07.487337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-64766ba8-53cc-4b26-bf5c-99c7fc49b5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:11:07.553472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 583/1179] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:11:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:11:07.825348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfa3de-c1bf-4a79-b5bf-53d90f293d6b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:07.925017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bfa3de-c1bf-4a79-b5bf-53d90f293d6b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:07.958452 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609934fd-5752-409d-a222-7513173b9f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:07.959078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609934fd-5752-409d-a222-7513173b9f52 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:07.959354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609934fd-5752-409d-a222-7513173b9f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:07.959624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609934fd-5752-409d-a222-7513173b9f52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:07.969879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609934fd-5752-409d-a222-7513173b9f52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-609934fd-5752-409d-a222-7513173b9f52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:11:07.974643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:11:07.975736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:11:07.976779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 597/1180] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:11:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:11:07.981857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64766ba8-53cc-4b26-bf5c-99c7fc49b5f4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:08.000250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:08.081799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64766ba8-53cc-4b26-bf5c-99c7fc49b5f4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:08.114640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04a105-191e-4fff-8882-3b5c55a8e664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:08.115231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04a105-191e-4fff-8882-3b5c55a8e664 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:08.115508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04a105-191e-4fff-8882-3b5c55a8e664 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:08.115774 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04a105-191e-4fff-8882-3b5c55a8e664 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:08.116426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04a105-191e-4fff-8882-3b5c55a8e664 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:11:08.116722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04a105-191e-4fff-8882-3b5c55a8e664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:08.118089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04a105-191e-4fff-8882-3b5c55a8e664 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:08.126314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 584/1181] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:11:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:08.132486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609934fd-5752-409d-a222-7513173b9f52 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:08.135351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:08.232020 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609934fd-5752-409d-a222-7513173b9f52 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:08.265446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1afa6c-28d2-45f7-9f34-a9baf3c86fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:08.266093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1afa6c-28d2-45f7-9f34-a9baf3c86fea None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:08.266376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1afa6c-28d2-45f7-9f34-a9baf3c86fea None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:08.266646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1afa6c-28d2-45f7-9f34-a9baf3c86fea None admin] PATH_INFO: `/v3/users/8b7ceee3c5ec473ca31f419ce41ae01c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:08.279374 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1afa6c-28d2-45f7-9f34-a9baf3c86fea None admin] RBAC: Authorizing `identity:get_user(user_id=8b7ceee3c5ec473ca31f419ce41ae01c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:08.280895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1afa6c-28d2-45f7-9f34-a9baf3c86fea None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:08.282541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 598/1182] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:11:08 2022] GET /identity/v3/users/8b7ceee3c5ec473ca31f419ce41ae01c => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:08.288390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04a105-191e-4fff-8882-3b5c55a8e664 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:08.372462 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:08.391675 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04a105-191e-4fff-8882-3b5c55a8e664 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:08.425774 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556587b7-ed51-47d8-b0ee-b47d8525ed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:08.426415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556587b7-ed51-47d8-b0ee-b47d8525ed46 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:08.426786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556587b7-ed51-47d8-b0ee-b47d8525ed46 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:08.427057 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556587b7-ed51-47d8-b0ee-b47d8525ed46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:08.495808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556587b7-ed51-47d8-b0ee-b47d8525ed46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:08.497260 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556587b7-ed51-47d8-b0ee-b47d8525ed46 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:08.574601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:08.595239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 585/1183] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:11:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:11:08.601206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1afa6c-28d2-45f7-9f34-a9baf3c86fea None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:08.659070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:08.698568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1afa6c-28d2-45f7-9f34-a9baf3c86fea None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:08.732558 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee099b-5f26-4a3e-a726-3005cbe761ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:08.733187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee099b-5f26-4a3e-a726-3005cbe761ca None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:08.733468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee099b-5f26-4a3e-a726-3005cbe761ca None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:08.733741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee099b-5f26-4a3e-a726-3005cbe761ca None admin] PATH_INFO: `/v3/projects/790f94919e79428ba4e5e41a34733fee` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:08.734899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee099b-5f26-4a3e-a726-3005cbe761ca None admin] RBAC: Authorizing `identity:get_project(project_id=790f94919e79428ba4e5e41a34733fee)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:08.736324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee099b-5f26-4a3e-a726-3005cbe761ca None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:08.737941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 599/1184] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:11:08 2022] GET /identity/v3/projects/790f94919e79428ba4e5e41a34733fee => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:08.744132 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556587b7-ed51-47d8-b0ee-b47d8525ed46 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:08.843840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556587b7-ed51-47d8-b0ee-b47d8525ed46 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:08.877870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17307ebe-66c4-4264-8407-6311bc4912a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:08.878493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17307ebe-66c4-4264-8407-6311bc4912a0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:08.878769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17307ebe-66c4-4264-8407-6311bc4912a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:08.879038 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17307ebe-66c4-4264-8407-6311bc4912a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:08.879757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17307ebe-66c4-4264-8407-6311bc4912a0 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=790f94919e79428ba4e5e41a34733fee, user.id=8b7ceee3c5ec473ca31f419ce41ae01c) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:11:08.880062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17307ebe-66c4-4264-8407-6311bc4912a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:08.881466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17307ebe-66c4-4264-8407-6311bc4912a0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:08.886788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 586/1185] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:11:08 2022] GET /identity/v3/role_assignments?user.id=8b7ceee3c5ec473ca31f419ce41ae01c&scope.project.id=790f94919e79428ba4e5e41a34733fee&role.id=459d01a218314543a7a8329e88b78331 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:11.022719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3ed3c-9b10-4b5d-944e-ddc968216bff None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:11.023293 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3ed3c-9b10-4b5d-944e-ddc968216bff None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:11.023293 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3ed3c-9b10-4b5d-944e-ddc968216bff None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:11.024574 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 600/1186] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:11:11.029725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735f8c3-d6bb-4225-a3d5-729ba3131481 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:11.030159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735f8c3-d6bb-4225-a3d5-729ba3131481 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:11.030274 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735f8c3-d6bb-4225-a3d5-729ba3131481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:11.047905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c735f8c3-d6bb-4225-a3d5-729ba3131481 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:11:11.049110 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c735f8c3-d6bb-4225-a3d5-729ba3131481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:11:11.089269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:11.117734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 587/1187] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:11:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:11:11.387043 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3ed3c-9b10-4b5d-944e-ddc968216bff None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:11.489489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3ed3c-9b10-4b5d-944e-ddc968216bff None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:11.518709 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:11.526847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873172a3-1111-46d8-9be1-e26553c6df7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:11.527476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873172a3-1111-46d8-9be1-e26553c6df7a None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:11.527758 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873172a3-1111-46d8-9be1-e26553c6df7a None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:11.528032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873172a3-1111-46d8-9be1-e26553c6df7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:11.538568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873172a3-1111-46d8-9be1-e26553c6df7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-873172a3-1111-46d8-9be1-e26553c6df7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:11:11.543386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:11:11.544540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:11:11.545629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:11:11.545629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:11:11.545629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:11:11.545629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:11:11.545629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:11:11.545629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:11:11.545629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:11:11.545629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:11:11.545629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:11:11.545629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:11:11.545629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:11:11.545629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:11:11.549530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 601/1188] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:11:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:11:11.550749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735f8c3-d6bb-4225-a3d5-729ba3131481 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:11.647944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735f8c3-d6bb-4225-a3d5-729ba3131481 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:11.679939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862ef80-4937-4e58-9612-13f0b48773e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:11.680538 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862ef80-4937-4e58-9612-13f0b48773e3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:11.680812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862ef80-4937-4e58-9612-13f0b48773e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:11.681079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862ef80-4937-4e58-9612-13f0b48773e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:11.681727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862ef80-4937-4e58-9612-13f0b48773e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:11:11.682027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862ef80-4937-4e58-9612-13f0b48773e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:11.683388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862ef80-4937-4e58-9612-13f0b48773e3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:11.691765 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 588/1189] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:11:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:11.697927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873172a3-1111-46d8-9be1-e26553c6df7a None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:11.795382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873172a3-1111-46d8-9be1-e26553c6df7a None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:11.829156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a841d-a508-44d7-bd97-b7ade2a55986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:11.829786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a841d-a508-44d7-bd97-b7ade2a55986 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:11.830085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a841d-a508-44d7-bd97-b7ade2a55986 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:11.830360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a841d-a508-44d7-bd97-b7ade2a55986 None admin] PATH_INFO: `/v3/users/8b7ceee3c5ec473ca31f419ce41ae01c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:11.831478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a841d-a508-44d7-bd97-b7ade2a55986 None admin] RBAC: Authorizing `identity:get_user(user_id=8b7ceee3c5ec473ca31f419ce41ae01c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:11.832890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a841d-a508-44d7-bd97-b7ade2a55986 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:11.834488 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 602/1190] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:11:11 2022] GET /identity/v3/users/8b7ceee3c5ec473ca31f419ce41ae01c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:11.840377 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862ef80-4937-4e58-9612-13f0b48773e3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:11.876164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:11.943723 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862ef80-4937-4e58-9612-13f0b48773e3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:11.976963 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2e77c-756b-4e26-ae47-03842f20a12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:11.977583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2e77c-756b-4e26-ae47-03842f20a12d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:11.977858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2e77c-756b-4e26-ae47-03842f20a12d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:11.978155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2e77c-756b-4e26-ae47-03842f20a12d None admin] PATH_INFO: `/v3/projects/790f94919e79428ba4e5e41a34733fee` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:11.979346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2e77c-756b-4e26-ae47-03842f20a12d None admin] RBAC: Authorizing `identity:get_project(project_id=790f94919e79428ba4e5e41a34733fee)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:11.980762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2e77c-756b-4e26-ae47-03842f20a12d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:11.982407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 589/1191] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:11:11 2022] GET /identity/v3/projects/790f94919e79428ba4e5e41a34733fee => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:11.988420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a841d-a508-44d7-bd97-b7ade2a55986 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:12.048733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:12.093311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a841d-a508-44d7-bd97-b7ade2a55986 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:12.125978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90c2eb-cfa0-47eb-93b8-36893291e483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:12.126662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90c2eb-cfa0-47eb-93b8-36893291e483 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:12.126941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90c2eb-cfa0-47eb-93b8-36893291e483 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:12.127209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90c2eb-cfa0-47eb-93b8-36893291e483 None admin] PATH_INFO: `/v3/projects/790f94919e79428ba4e5e41a34733fee/users/8b7ceee3c5ec473ca31f419ce41ae01c/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:12.129079 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90c2eb-cfa0-47eb-93b8-36893291e483 None admin] RBAC: Authorizing `identity:create_grant(project_id=790f94919e79428ba4e5e41a34733fee, user_id=8b7ceee3c5ec473ca31f419ce41ae01c, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:12.130534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90c2eb-cfa0-47eb-93b8-36893291e483 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:12.143720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 603/1192] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:11:11 2022] PUT /identity/v3/projects/790f94919e79428ba4e5e41a34733fee/users/8b7ceee3c5ec473ca31f419ce41ae01c/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:11:14.262328 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0886a2-5f51-4431-b2de-e8e1ebc0609a None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:14.262860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0886a2-5f51-4431-b2de-e8e1ebc0609a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:14.262948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0886a2-5f51-4431-b2de-e8e1ebc0609a None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:14.264218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 590/1193] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:11:14.270066 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134d32d-b800-4d36-b0ef-ad46206666ea None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:14.270470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134d32d-b800-4d36-b0ef-ad46206666ea None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:14.270617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134d32d-b800-4d36-b0ef-ad46206666ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:14.289694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3134d32d-b800-4d36-b0ef-ad46206666ea None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:11:14.290917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3134d32d-b800-4d36-b0ef-ad46206666ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:11:14.356913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 604/1194] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:11:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:11:14.625962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0886a2-5f51-4431-b2de-e8e1ebc0609a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:14.725791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0886a2-5f51-4431-b2de-e8e1ebc0609a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:14.728084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:14.759372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917ce77-fa4d-4eb7-b745-3249ff3f798e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:14.759997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917ce77-fa4d-4eb7-b745-3249ff3f798e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:14.760276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917ce77-fa4d-4eb7-b745-3249ff3f798e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:14.760546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917ce77-fa4d-4eb7-b745-3249ff3f798e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:14.771000 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917ce77-fa4d-4eb7-b745-3249ff3f798e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d917ce77-fa4d-4eb7-b745-3249ff3f798e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:11:14.775777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:11:14.776863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:11:14.778111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 591/1195] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:11:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:11:14.782936 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134d32d-b800-4d36-b0ef-ad46206666ea None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:14.828757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:14.890104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134d32d-b800-4d36-b0ef-ad46206666ea None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:14.922651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bf5d6-9dd0-4432-90d3-b931ec1de928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:14.923255 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bf5d6-9dd0-4432-90d3-b931ec1de928 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:14.923533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bf5d6-9dd0-4432-90d3-b931ec1de928 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:14.923805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bf5d6-9dd0-4432-90d3-b931ec1de928 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:14.924466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bf5d6-9dd0-4432-90d3-b931ec1de928 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:11:14.924770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bf5d6-9dd0-4432-90d3-b931ec1de928 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:14.926143 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bf5d6-9dd0-4432-90d3-b931ec1de928 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:14.933929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 605/1196] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:11:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:14.940455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917ce77-fa4d-4eb7-b745-3249ff3f798e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:15.040244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917ce77-fa4d-4eb7-b745-3249ff3f798e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:15.074633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf3c0d-ee7b-468b-9d29-dfa6e8ecfe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:15.075253 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf3c0d-ee7b-468b-9d29-dfa6e8ecfe9a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:15.075533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf3c0d-ee7b-468b-9d29-dfa6e8ecfe9a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:15.075807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf3c0d-ee7b-468b-9d29-dfa6e8ecfe9a None admin] PATH_INFO: `/v3/users/8b7ceee3c5ec473ca31f419ce41ae01c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:15.076968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf3c0d-ee7b-468b-9d29-dfa6e8ecfe9a None admin] RBAC: Authorizing `identity:get_user(user_id=8b7ceee3c5ec473ca31f419ce41ae01c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:15.078421 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf3c0d-ee7b-468b-9d29-dfa6e8ecfe9a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:15.080000 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 592/1197] 10.222.0.64 () {56 vars in 1177 bytes} [Wed Sep 21 16:11:14 2022] GET /identity/v3/users/8b7ceee3c5ec473ca31f419ce41ae01c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:15.085869 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bf5d6-9dd0-4432-90d3-b931ec1de928 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:15.192082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bf5d6-9dd0-4432-90d3-b931ec1de928 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:15.201008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:15.225056 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388843e-4bb6-4b30-9828-1a3c91e48079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:15.225660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388843e-4bb6-4b30-9828-1a3c91e48079 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:15.225938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388843e-4bb6-4b30-9828-1a3c91e48079 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:15.226238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388843e-4bb6-4b30-9828-1a3c91e48079 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:15.292872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388843e-4bb6-4b30-9828-1a3c91e48079 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:15.294339 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388843e-4bb6-4b30-9828-1a3c91e48079 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:15.392521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 606/1198] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:11:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:11:15.398505 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf3c0d-ee7b-468b-9d29-dfa6e8ecfe9a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:15.429855 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:15.497485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf3c0d-ee7b-468b-9d29-dfa6e8ecfe9a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:15.530894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fa6f4-4883-4e5d-b9b2-b0ee41ad5050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:15.531524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fa6f4-4883-4e5d-b9b2-b0ee41ad5050 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:15.531804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fa6f4-4883-4e5d-b9b2-b0ee41ad5050 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:15.532076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fa6f4-4883-4e5d-b9b2-b0ee41ad5050 None admin] PATH_INFO: `/v3/projects/790f94919e79428ba4e5e41a34733fee` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:15.533179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fa6f4-4883-4e5d-b9b2-b0ee41ad5050 None admin] RBAC: Authorizing `identity:get_project(project_id=790f94919e79428ba4e5e41a34733fee)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:15.534623 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fa6f4-4883-4e5d-b9b2-b0ee41ad5050 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:15.536235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 593/1199] 10.222.0.64 () {56 vars in 1186 bytes} [Wed Sep 21 16:11:15 2022] GET /identity/v3/projects/790f94919e79428ba4e5e41a34733fee => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:15.542390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388843e-4bb6-4b30-9828-1a3c91e48079 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:11:15.552953 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:15.641356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388843e-4bb6-4b30-9828-1a3c91e48079 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:11:15.679078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09723fdf-c08b-49e3-87d3-3355a0c42600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:11:15.679682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09723fdf-c08b-49e3-87d3-3355a0c42600 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:11:15.679964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09723fdf-c08b-49e3-87d3-3355a0c42600 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:11:15.680237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09723fdf-c08b-49e3-87d3-3355a0c42600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:11:15.680963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09723fdf-c08b-49e3-87d3-3355a0c42600 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=790f94919e79428ba4e5e41a34733fee, user.id=8b7ceee3c5ec473ca31f419ce41ae01c) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:11:15.681274 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09723fdf-c08b-49e3-87d3-3355a0c42600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:11:15.682718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09723fdf-c08b-49e3-87d3-3355a0c42600 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:11:15.688537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 607/1200] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:11:15 2022] GET /identity/v3/role_assignments?user.id=8b7ceee3c5ec473ca31f419ce41ae01c&scope.project.id=790f94919e79428ba4e5e41a34733fee&role.id=459d01a218314543a7a8329e88b78331 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:11:25.526311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=50 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:11:25.686447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=64 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:13.534870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befd52e-ecbc-4e29-8c3c-1329b8e8a607 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:13.535942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befd52e-ecbc-4e29-8c3c-1329b8e8a607 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:13.535942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3befd52e-ecbc-4e29-8c3c-1329b8e8a607 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:13.537388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 594/1201] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:12:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:12:13.548570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff22a1-48bb-4f1f-b66c-c0b57dfae2e5 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:13.549320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff22a1-48bb-4f1f-b66c-c0b57dfae2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:13.549320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff22a1-48bb-4f1f-b66c-c0b57dfae2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:13.583583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f7ff22a1-48bb-4f1f-b66c-c0b57dfae2e5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:12:13.585682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f7ff22a1-48bb-4f1f-b66c-c0b57dfae2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:12:13.719999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 608/1202] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:12:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:12:14.121146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befd52e-ecbc-4e29-8c3c-1329b8e8a607 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:14.255846 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3befd52e-ecbc-4e29-8c3c-1329b8e8a607 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:14.321293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ebc84-2ed7-4e51-937e-edadba32d5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:14.321613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ebc84-2ed7-4e51-937e-edadba32d5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:14.321816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ebc84-2ed7-4e51-937e-edadba32d5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:14.322118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ebc84-2ed7-4e51-937e-edadba32d5e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:14.335803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ebc84-2ed7-4e51-937e-edadba32d5e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:14.337105 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ebc84-2ed7-4e51-937e-edadba32d5e7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-439ebc84-2ed7-4e51-937e-edadba32d5e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:14.357231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:14.359566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:14.361437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:14.361437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:14.361437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:14.364129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 595/1203] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:14.365068 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff22a1-48bb-4f1f-b66c-c0b57dfae2e5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:14.582193 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff22a1-48bb-4f1f-b66c-c0b57dfae2e5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:14.636367 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:14.646004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1e624-b5d0-4fca-bb0a-4e5ab7aee85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:14.647137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1e624-b5d0-4fca-bb0a-4e5ab7aee85b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:14.647653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1e624-b5d0-4fca-bb0a-4e5ab7aee85b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:14.648162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1e624-b5d0-4fca-bb0a-4e5ab7aee85b None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:14.649409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1e624-b5d0-4fca-bb0a-4e5ab7aee85b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:14.649927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1e624-b5d0-4fca-bb0a-4e5ab7aee85b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:14.652222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1e624-b5d0-4fca-bb0a-4e5ab7aee85b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:14.678540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 609/1204] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:12:14.683231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ebc84-2ed7-4e51-937e-edadba32d5e7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:14.798484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:14.912747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ebc84-2ed7-4e51-937e-edadba32d5e7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:15.001033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a76546-a753-44ef-83ed-efa169631f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:15.002816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a76546-a753-44ef-83ed-efa169631f06 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:15.003661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a76546-a753-44ef-83ed-efa169631f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:15.004468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a76546-a753-44ef-83ed-efa169631f06 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:15.006389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a76546-a753-44ef-83ed-efa169631f06 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:15.009010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a76546-a753-44ef-83ed-efa169631f06 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:15.062753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 596/1205] 10.222.0.64 () {60 vars in 1132 bytes} [Wed Sep 21 16:12:14 2022] POST /identity/v3/users => generated 251 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:12:18.026558 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d1207-41ed-47ff-8d93-78e610f88fb7 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:18.026558 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d1207-41ed-47ff-8d93-78e610f88fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:18.028014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d1207-41ed-47ff-8d93-78e610f88fb7 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:18.028675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 610/1206] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:12:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:12:18.042098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f66ac-af0b-417c-adac-b28339bd2e43 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:18.042098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f66ac-af0b-417c-adac-b28339bd2e43 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:18.042098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f66ac-af0b-417c-adac-b28339bd2e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:18.066689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f14f66ac-af0b-417c-adac-b28339bd2e43 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:12:18.067928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f14f66ac-af0b-417c-adac-b28339bd2e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:12:18.164256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 597/1207] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:12:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:12:18.748909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d1207-41ed-47ff-8d93-78e610f88fb7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:18.921409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d1207-41ed-47ff-8d93-78e610f88fb7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:18.985060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5f0f0-71c6-4437-bccd-195165ccb887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:18.986002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5f0f0-71c6-4437-bccd-195165ccb887 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:18.986483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5f0f0-71c6-4437-bccd-195165ccb887 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:18.986974 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5f0f0-71c6-4437-bccd-195165ccb887 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:19.002187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5f0f0-71c6-4437-bccd-195165ccb887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-92f5f0f0-71c6-4437-bccd-195165ccb887 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:12:19.007200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:19.009238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:19.010832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 611/1208] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:19.016607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f66ac-af0b-417c-adac-b28339bd2e43 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:19.083214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:19.188546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f66ac-af0b-417c-adac-b28339bd2e43 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:19.246736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1181515-3bc0-493c-bba9-e7fb830d31a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:19.247527 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1181515-3bc0-493c-bba9-e7fb830d31a6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:19.247979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1181515-3bc0-493c-bba9-e7fb830d31a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:19.248307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1181515-3bc0-493c-bba9-e7fb830d31a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:19.249124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1181515-3bc0-493c-bba9-e7fb830d31a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:19.249456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1181515-3bc0-493c-bba9-e7fb830d31a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:19.251354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1181515-3bc0-493c-bba9-e7fb830d31a6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:19.270495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 598/1209] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:12:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:19.273351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5f0f0-71c6-4437-bccd-195165ccb887 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:19.354274 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:19.422579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5f0f0-71c6-4437-bccd-195165ccb887 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:19.467341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d72614-24a8-4fb0-a6b6-9d7622b965cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:19.467999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d72614-24a8-4fb0-a6b6-9d7622b965cc None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:19.468271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d72614-24a8-4fb0-a6b6-9d7622b965cc None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:19.468560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d72614-24a8-4fb0-a6b6-9d7622b965cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:19.489502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d72614-24a8-4fb0-a6b6-9d7622b965cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:19.491986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d72614-24a8-4fb0-a6b6-9d7622b965cc None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-38d72614-24a8-4fb0-a6b6-9d7622b965cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:19.514097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:19.515571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:19.516870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:19.516870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:19.516870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:19.524166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 612/1210] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:19.527613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1181515-3bc0-493c-bba9-e7fb830d31a6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:19.672982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:19.677415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1181515-3bc0-493c-bba9-e7fb830d31a6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:19.738770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03b63a-b2be-4d62-948a-126135cd1cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:19.739604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03b63a-b2be-4d62-948a-126135cd1cef None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:19.740025 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03b63a-b2be-4d62-948a-126135cd1cef None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:19.740401 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03b63a-b2be-4d62-948a-126135cd1cef None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:19.741387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03b63a-b2be-4d62-948a-126135cd1cef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:19.741702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03b63a-b2be-4d62-948a-126135cd1cef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:19.744639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03b63a-b2be-4d62-948a-126135cd1cef None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:19.761537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 599/1211] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:19.770741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d72614-24a8-4fb0-a6b6-9d7622b965cc None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:19.953674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d72614-24a8-4fb0-a6b6-9d7622b965cc None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:19.957179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:20.030891 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f1b8e-6fc9-4f5a-8c81-42839d70d0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:20.032092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f1b8e-6fc9-4f5a-8c81-42839d70d0ef None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:20.032593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f1b8e-6fc9-4f5a-8c81-42839d70d0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:20.032952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f1b8e-6fc9-4f5a-8c81-42839d70d0ef None admin] PATH_INFO: `/v3/users/placement` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:20.053559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f1b8e-6fc9-4f5a-8c81-42839d70d0ef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:20.057123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f1b8e-6fc9-4f5a-8c81-42839d70d0ef None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-5f6f1b8e-6fc9-4f5a-8c81-42839d70d0ef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:12:20.076019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:20.078163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:20.088764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:20.088764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:20.088764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:12:20.088764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:12:20.088764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:12:20.088764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:12:20.088764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:12:20.088764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:12:20.088764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 16:12:20.088764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:20.088764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 613/1212] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:20.092424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03b63a-b2be-4d62-948a-126135cd1cef None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:20.268128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03b63a-b2be-4d62-948a-126135cd1cef None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:20.320603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd8c55-79df-421f-9749-dbcce233ea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:20.322287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd8c55-79df-421f-9749-dbcce233ea9c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:20.322699 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd8c55-79df-421f-9749-dbcce233ea9c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:20.322982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd8c55-79df-421f-9749-dbcce233ea9c None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:20.323768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd8c55-79df-421f-9749-dbcce233ea9c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:20.324100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd8c55-79df-421f-9749-dbcce233ea9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:20.325606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd8c55-79df-421f-9749-dbcce233ea9c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:20.355586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 600/1213] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Sep 21 16:12:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:20.362067 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f1b8e-6fc9-4f5a-8c81-42839d70d0ef None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:20.546214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f1b8e-6fc9-4f5a-8c81-42839d70d0ef None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:20.608926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d51e43-ec5a-4145-9286-2ecebe5c2042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:20.610457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d51e43-ec5a-4145-9286-2ecebe5c2042 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:20.610750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d51e43-ec5a-4145-9286-2ecebe5c2042 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:20.611016 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d51e43-ec5a-4145-9286-2ecebe5c2042 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:20.652823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:20.718704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d51e43-ec5a-4145-9286-2ecebe5c2042 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:20.721878 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d51e43-ec5a-4145-9286-2ecebe5c2042 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:20.873640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 614/1214] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:12:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:12:20.882643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd8c55-79df-421f-9749-dbcce233ea9c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:20.911362 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:21.058253 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd8c55-79df-421f-9749-dbcce233ea9c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:21.108409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5149e42-7285-4715-8c5c-9fad524d7cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:21.110278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5149e42-7285-4715-8c5c-9fad524d7cbb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:21.110672 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5149e42-7285-4715-8c5c-9fad524d7cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:21.111031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5149e42-7285-4715-8c5c-9fad524d7cbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:21.127589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5149e42-7285-4715-8c5c-9fad524d7cbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:21.129440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5149e42-7285-4715-8c5c-9fad524d7cbb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-e5149e42-7285-4715-8c5c-9fad524d7cbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:21.154472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:21.155700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:21.156838 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:21.156838 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:21.156838 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:21.157950 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 601/1215] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:21.164265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d51e43-ec5a-4145-9286-2ecebe5c2042 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:21.250845 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:21.316741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d51e43-ec5a-4145-9286-2ecebe5c2042 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:21.364088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ce56a-6d9c-4039-b759-def805e0b8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:21.365082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ce56a-6d9c-4039-b759-def805e0b8de None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:21.365082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ce56a-6d9c-4039-b759-def805e0b8de None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:21.365398 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ce56a-6d9c-4039-b759-def805e0b8de None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:21.366067 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ce56a-6d9c-4039-b759-def805e0b8de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:21.366380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ce56a-6d9c-4039-b759-def805e0b8de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:21.368185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ce56a-6d9c-4039-b759-def805e0b8de None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:21.386161 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 615/1216] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:21.392966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5149e42-7285-4715-8c5c-9fad524d7cbb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:21.487897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:21.547759 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5149e42-7285-4715-8c5c-9fad524d7cbb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:21.620535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1241768-3ab9-4976-9853-478ee8bade01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:21.621322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1241768-3ab9-4976-9853-478ee8bade01 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:21.621785 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1241768-3ab9-4976-9853-478ee8bade01 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:21.622257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1241768-3ab9-4976-9853-478ee8bade01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:21.635478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1241768-3ab9-4976-9853-478ee8bade01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:21.638296 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1241768-3ab9-4976-9853-478ee8bade01 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c1241768-3ab9-4976-9853-478ee8bade01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:12:21.663905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:21.666754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:21.668820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:12:21.668820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:12:21.668820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:12:21.668820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:12:21.668820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:12:21.668820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:12:21.668820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:12:21.668820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:21.668820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 602/1217] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:21.679462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ce56a-6d9c-4039-b759-def805e0b8de None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:21.863200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ce56a-6d9c-4039-b759-def805e0b8de None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:21.914427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:21.937757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd111598-67eb-45db-9133-b39bd93de489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:21.938743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd111598-67eb-45db-9133-b39bd93de489 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:21.939026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd111598-67eb-45db-9133-b39bd93de489 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:21.939330 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd111598-67eb-45db-9133-b39bd93de489 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:21.940765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd111598-67eb-45db-9133-b39bd93de489 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:21.941415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd111598-67eb-45db-9133-b39bd93de489 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:21.943406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd111598-67eb-45db-9133-b39bd93de489 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:21.959995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 616/1218] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:21.967517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1241768-3ab9-4976-9853-478ee8bade01 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:22.128131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1241768-3ab9-4976-9853-478ee8bade01 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:22.157973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:22.164160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4f6fb-a5ec-40f1-8685-4438d21510c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:22.164816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4f6fb-a5ec-40f1-8685-4438d21510c4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:22.165047 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4f6fb-a5ec-40f1-8685-4438d21510c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:22.165356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4f6fb-a5ec-40f1-8685-4438d21510c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:22.166670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4f6fb-a5ec-40f1-8685-4438d21510c4 None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=55642294e8ff4f779510f4646e0e30a8) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:22.166975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4f6fb-a5ec-40f1-8685-4438d21510c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:22.168390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4f6fb-a5ec-40f1-8685-4438d21510c4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:22.174073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 603/1219] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:12:21 2022] GET /identity/v3/role_assignments?user.id=55642294e8ff4f779510f4646e0e30a8&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:24.518937 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a691e-42d6-44be-a285-27788b8f9a76 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:24.519482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a691e-42d6-44be-a285-27788b8f9a76 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:24.519874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a691e-42d6-44be-a285-27788b8f9a76 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:24.522366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 617/1220] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:12:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:12:24.528559 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13589bb-42b9-4e33-a0d3-5128f961ab85 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:24.529315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13589bb-42b9-4e33-a0d3-5128f961ab85 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:24.529315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13589bb-42b9-4e33-a0d3-5128f961ab85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:24.550927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c13589bb-42b9-4e33-a0d3-5128f961ab85 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:12:24.552494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c13589bb-42b9-4e33-a0d3-5128f961ab85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:12:24.636708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 604/1221] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:12:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:12:24.924663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a691e-42d6-44be-a285-27788b8f9a76 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:25.067281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a691e-42d6-44be-a285-27788b8f9a76 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:25.112385 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4059b-d2c4-485c-900b-eaf77a62033b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:25.113155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4059b-d2c4-485c-900b-eaf77a62033b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:25.113485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4059b-d2c4-485c-900b-eaf77a62033b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:25.113771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4059b-d2c4-485c-900b-eaf77a62033b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:25.131643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4059b-d2c4-485c-900b-eaf77a62033b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-53b4059b-d2c4-485c-900b-eaf77a62033b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:12:25.141408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:25.142567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:25.143816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 618/1222] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:12:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:25.150254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13589bb-42b9-4e33-a0d3-5128f961ab85 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:25.303748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13589bb-42b9-4e33-a0d3-5128f961ab85 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:25.331748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:25.348278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ad2d0-7c42-4b0d-bfb8-2aa83597601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:25.348912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ad2d0-7c42-4b0d-bfb8-2aa83597601e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:25.349193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ad2d0-7c42-4b0d-bfb8-2aa83597601e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:25.349464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ad2d0-7c42-4b0d-bfb8-2aa83597601e None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:25.350183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ad2d0-7c42-4b0d-bfb8-2aa83597601e None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:25.350482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ad2d0-7c42-4b0d-bfb8-2aa83597601e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:25.351928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ad2d0-7c42-4b0d-bfb8-2aa83597601e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:25.361687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 605/1223] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:12:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:25.369099 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4059b-d2c4-485c-900b-eaf77a62033b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:25.430735 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:25.532219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4059b-d2c4-485c-900b-eaf77a62033b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:25.572058 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87eccd5-6cb4-44d3-a539-582dc0e77bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:25.572614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eccd5-6cb4-44d3-a539-582dc0e77bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:25.572885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eccd5-6cb4-44d3-a539-582dc0e77bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:25.573154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eccd5-6cb4-44d3-a539-582dc0e77bd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:25.586211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87eccd5-6cb4-44d3-a539-582dc0e77bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:25.588303 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87eccd5-6cb4-44d3-a539-582dc0e77bd2 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-a87eccd5-6cb4-44d3-a539-582dc0e77bd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:25.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:25.609724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:25.610841 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:25.610841 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:25.610841 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:25.610841 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 619/1224] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:12:25.615605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ad2d0-7c42-4b0d-bfb8-2aa83597601e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:25.744111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ad2d0-7c42-4b0d-bfb8-2aa83597601e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:25.796874 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bf627-f657-40fe-b401-fd92814ac7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:25.797585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bf627-f657-40fe-b401-fd92814ac7ea None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:25.797873 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bf627-f657-40fe-b401-fd92814ac7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:25.798274 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bf627-f657-40fe-b401-fd92814ac7ea None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:25.799137 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bf627-f657-40fe-b401-fd92814ac7ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:25.799478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bf627-f657-40fe-b401-fd92814ac7ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:25.802262 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bf627-f657-40fe-b401-fd92814ac7ea None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:25.822264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 606/1225] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:25.827854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87eccd5-6cb4-44d3-a539-582dc0e77bd2 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:25.957114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:25.965091 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87eccd5-6cb4-44d3-a539-582dc0e77bd2 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:26.012426 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb172c-6842-4230-ac2d-80c97cd1345f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:26.013089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb172c-6842-4230-ac2d-80c97cd1345f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:26.013374 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb172c-6842-4230-ac2d-80c97cd1345f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:26.013689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbb172c-6842-4230-ac2d-80c97cd1345f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:26.029556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb172c-6842-4230-ac2d-80c97cd1345f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:26.031334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbb172c-6842-4230-ac2d-80c97cd1345f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-0dbb172c-6842-4230-ac2d-80c97cd1345f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:12:26.046754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:26.047901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:26.048995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:26.048995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:26.048995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:12:26.048995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:12:26.048995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:12:26.048995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:12:26.048995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:12:26.048995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:12:26.048995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 16:12:26.048995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:26.048995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 620/1226] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:12:26.060577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bf627-f657-40fe-b401-fd92814ac7ea None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:26.123449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:26.193570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bf627-f657-40fe-b401-fd92814ac7ea None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:26.242457 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce5208-018b-4d37-8435-4a4edebce595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:26.243223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce5208-018b-4d37-8435-4a4edebce595 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:26.244215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce5208-018b-4d37-8435-4a4edebce595 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:26.244680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce5208-018b-4d37-8435-4a4edebce595 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:26.246176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce5208-018b-4d37-8435-4a4edebce595 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:26.247962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce5208-018b-4d37-8435-4a4edebce595 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:26.250504 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce5208-018b-4d37-8435-4a4edebce595 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:26.270901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 607/1227] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Sep 21 16:12:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:26.283645 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb172c-6842-4230-ac2d-80c97cd1345f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:26.416155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbb172c-6842-4230-ac2d-80c97cd1345f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:26.456899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60fc4c-3c9f-4b09-93bc-a08d93904b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:26.457950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60fc4c-3c9f-4b09-93bc-a08d93904b79 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:26.458392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60fc4c-3c9f-4b09-93bc-a08d93904b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:26.458748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d60fc4c-3c9f-4b09-93bc-a08d93904b79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:26.473496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60fc4c-3c9f-4b09-93bc-a08d93904b79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:26.475055 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d60fc4c-3c9f-4b09-93bc-a08d93904b79 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-8d60fc4c-3c9f-4b09-93bc-a08d93904b79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:26.503294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:26.509527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:26.510682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:26.510682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:26.510682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:26.510682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 621/1228] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:26.515168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce5208-018b-4d37-8435-4a4edebce595 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:26.663534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce5208-018b-4d37-8435-4a4edebce595 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:26.667805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:26.716206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fafad-bff2-4e80-8873-81832ed7c320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:26.716893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fafad-bff2-4e80-8873-81832ed7c320 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:26.717375 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fafad-bff2-4e80-8873-81832ed7c320 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:26.717769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fafad-bff2-4e80-8873-81832ed7c320 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:26.718459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fafad-bff2-4e80-8873-81832ed7c320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:26.718762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fafad-bff2-4e80-8873-81832ed7c320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:26.720120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fafad-bff2-4e80-8873-81832ed7c320 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:26.733399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 608/1229] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:26.740361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60fc4c-3c9f-4b09-93bc-a08d93904b79 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:26.750717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:26.859209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d60fc4c-3c9f-4b09-93bc-a08d93904b79 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:26.911754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30709f-10f6-422c-9b29-d44ca8709f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:26.912745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30709f-10f6-422c-9b29-d44ca8709f4d None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:26.913236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30709f-10f6-422c-9b29-d44ca8709f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:26.913711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30709f-10f6-422c-9b29-d44ca8709f4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:26.923797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30709f-10f6-422c-9b29-d44ca8709f4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:26.925175 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30709f-10f6-422c-9b29-d44ca8709f4d None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-fa30709f-10f6-422c-9b29-d44ca8709f4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:12:26.941530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:26.942708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:26.943806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:12:26.943806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:12:26.943806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:12:26.943806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:12:26.943806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:12:26.943806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:12:26.943806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:12:26.943806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:26.943806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 622/1230] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:12:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:12:26.953080 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fafad-bff2-4e80-8873-81832ed7c320 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:27.115449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fafad-bff2-4e80-8873-81832ed7c320 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:27.165900 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049ae90-192b-40ff-bbd0-18b39992413f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:27.166561 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049ae90-192b-40ff-bbd0-18b39992413f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:27.166841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049ae90-192b-40ff-bbd0-18b39992413f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:27.167109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049ae90-192b-40ff-bbd0-18b39992413f None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:27.167822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049ae90-192b-40ff-bbd0-18b39992413f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:27.168251 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049ae90-192b-40ff-bbd0-18b39992413f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:27.169804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049ae90-192b-40ff-bbd0-18b39992413f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:27.185873 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 609/1231] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:12:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:27.192839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30709f-10f6-422c-9b29-d44ca8709f4d None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:27.273646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:27.320808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30709f-10f6-422c-9b29-d44ca8709f4d None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:27.367432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a7024e-b042-4412-8401-2abe06047b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:27.368658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7024e-b042-4412-8401-2abe06047b37 None admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:27.368941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7024e-b042-4412-8401-2abe06047b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:27.369216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a7024e-b042-4412-8401-2abe06047b37 None admin] PATH_INFO: `/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/55642294e8ff4f779510f4646e0e30a8/roles/5f8e9da552194885bbd7684f1a9c4f2d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:27.385722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7024e-b042-4412-8401-2abe06047b37 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9e1d9331b4d4dbd3dd447937c42ad, user_id=55642294e8ff4f779510f4646e0e30a8, role_id=5f8e9da552194885bbd7684f1a9c4f2d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:27.387414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a7024e-b042-4412-8401-2abe06047b37 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:27.410090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 623/1232] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:12:27 2022] PUT /identity/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/55642294e8ff4f779510f4646e0e30a8/roles/5f8e9da552194885bbd7684f1a9c4f2d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:12:29.991223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95e989-a12f-4604-add1-4c7490207d49 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:29.991834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95e989-a12f-4604-add1-4c7490207d49 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:29.991957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95e989-a12f-4604-add1-4c7490207d49 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:29.993917 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 610/1233] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:12:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:12:29.999861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182e9fd-48d3-49a9-8b1b-323a7fe08fe8 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:30.000668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182e9fd-48d3-49a9-8b1b-323a7fe08fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:30.000668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182e9fd-48d3-49a9-8b1b-323a7fe08fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:30.020725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1182e9fd-48d3-49a9-8b1b-323a7fe08fe8 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:12:30.021973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1182e9fd-48d3-49a9-8b1b-323a7fe08fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:12:30.117967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 624/1234] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:12:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 21 16:12:30.401667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95e989-a12f-4604-add1-4c7490207d49 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:30.431146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:30.532492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea95e989-a12f-4604-add1-4c7490207d49 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:30.578193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd495a09-0912-479d-9fe2-7401260d780c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:30.578861 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd495a09-0912-479d-9fe2-7401260d780c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:30.579141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd495a09-0912-479d-9fe2-7401260d780c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:30.579420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd495a09-0912-479d-9fe2-7401260d780c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:30.596197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd495a09-0912-479d-9fe2-7401260d780c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-dd495a09-0912-479d-9fe2-7401260d780c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:12:30.601052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:30.602307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:30.603171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 611/1235] 10.222.0.64 () {56 vars in 1102 bytes} [Wed Sep 21 16:12:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:30.608385 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182e9fd-48d3-49a9-8b1b-323a7fe08fe8 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:30.668487 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:30.737534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182e9fd-48d3-49a9-8b1b-323a7fe08fe8 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:30.784483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e6dbf-be2f-49fd-b7cd-00ad5bc0564f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:30.785103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e6dbf-be2f-49fd-b7cd-00ad5bc0564f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:30.785381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e6dbf-be2f-49fd-b7cd-00ad5bc0564f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:30.785670 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e6dbf-be2f-49fd-b7cd-00ad5bc0564f None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:30.786360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e6dbf-be2f-49fd-b7cd-00ad5bc0564f None admin] RBAC: Adding query filter params (name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:30.786664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e6dbf-be2f-49fd-b7cd-00ad5bc0564f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:30.788146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e6dbf-be2f-49fd-b7cd-00ad5bc0564f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:30.798601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 625/1236] 10.222.0.64 () {56 vars in 1103 bytes} [Wed Sep 21 16:12:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:30.809022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd495a09-0912-479d-9fe2-7401260d780c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:30.931608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:30.953184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd495a09-0912-479d-9fe2-7401260d780c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:31.000467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda9c86-cab1-49d9-ae0b-7f76561a671a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:31.001259 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda9c86-cab1-49d9-ae0b-7f76561a671a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:31.001567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda9c86-cab1-49d9-ae0b-7f76561a671a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:31.001870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda9c86-cab1-49d9-ae0b-7f76561a671a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:31.014592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda9c86-cab1-49d9-ae0b-7f76561a671a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:31.016470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda9c86-cab1-49d9-ae0b-7f76561a671a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0cda9c86-cab1-49d9-ae0b-7f76561a671a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:31.040653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:31.043018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:31.044912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:31.044912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:31.044912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:31.044912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 612/1237] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:31.049582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e6dbf-be2f-49fd-b7cd-00ad5bc0564f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:31.204320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e6dbf-be2f-49fd-b7cd-00ad5bc0564f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:31.221922 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:31.255826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bb885-c177-495a-b142-7a2959cb84ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:31.256457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bb885-c177-495a-b142-7a2959cb84ae None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:31.256792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bb885-c177-495a-b142-7a2959cb84ae None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:31.257136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bb885-c177-495a-b142-7a2959cb84ae None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:31.257932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bb885-c177-495a-b142-7a2959cb84ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:31.258245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bb885-c177-495a-b142-7a2959cb84ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:31.260351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bb885-c177-495a-b142-7a2959cb84ae None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:31.275490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 626/1238] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:31.281484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda9c86-cab1-49d9-ae0b-7f76561a671a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:31.409328 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda9c86-cab1-49d9-ae0b-7f76561a671a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:31.452908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b29428-8c92-4ae6-b2ee-4dc67aef7e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:31.453842 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b29428-8c92-4ae6-b2ee-4dc67aef7e08 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:31.454293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b29428-8c92-4ae6-b2ee-4dc67aef7e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:31.454788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b29428-8c92-4ae6-b2ee-4dc67aef7e08 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:31.458556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:31.465715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b29428-8c92-4ae6-b2ee-4dc67aef7e08 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:31.467231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b29428-8c92-4ae6-b2ee-4dc67aef7e08 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-09b29428-8c92-4ae6-b2ee-4dc67aef7e08 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:12:31.482583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:31.483947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:31.484973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:31.484973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:31.484973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:12:31.484973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:12:31.484973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:12:31.484973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:12:31.484973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:12:31.484973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:12:31.484973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 16:12:31.484973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:31.484973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 613/1239] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:12:31.490632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bb885-c177-495a-b142-7a2959cb84ae None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:31.635088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bb885-c177-495a-b142-7a2959cb84ae None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:31.683501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65ad7a-b317-4fae-b9c8-db17d2c453a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:31.684186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65ad7a-b317-4fae-b9c8-db17d2c453a1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:31.684457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65ad7a-b317-4fae-b9c8-db17d2c453a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:31.684801 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65ad7a-b317-4fae-b9c8-db17d2c453a1 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:31.685505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65ad7a-b317-4fae-b9c8-db17d2c453a1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:31.685763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65ad7a-b317-4fae-b9c8-db17d2c453a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:31.687581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65ad7a-b317-4fae-b9c8-db17d2c453a1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:31.720922 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 627/1240] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Sep 21 16:12:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:31.726986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b29428-8c92-4ae6-b2ee-4dc67aef7e08 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:31.854139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b29428-8c92-4ae6-b2ee-4dc67aef7e08 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:31.899646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad5761-7a46-4241-9f1c-ecf0e585cd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:31.900383 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad5761-7a46-4241-9f1c-ecf0e585cd05 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:31.900858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad5761-7a46-4241-9f1c-ecf0e585cd05 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:31.901214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad5761-7a46-4241-9f1c-ecf0e585cd05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:31.978357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:31.985770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad5761-7a46-4241-9f1c-ecf0e585cd05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:31.987321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad5761-7a46-4241-9f1c-ecf0e585cd05 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:32.106399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 614/1241] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:12:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:12:32.115136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65ad7a-b317-4fae-b9c8-db17d2c453a1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:32.151876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:32.247275 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65ad7a-b317-4fae-b9c8-db17d2c453a1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:32.297562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f390991-6f49-40bd-9766-a77a71a50467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:32.298278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f390991-6f49-40bd-9766-a77a71a50467 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:32.298586 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f390991-6f49-40bd-9766-a77a71a50467 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:32.298907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f390991-6f49-40bd-9766-a77a71a50467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:32.319390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f390991-6f49-40bd-9766-a77a71a50467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:32.321805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f390991-6f49-40bd-9766-a77a71a50467 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6f390991-6f49-40bd-9766-a77a71a50467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:32.339473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:32.342830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:32.344075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:32.344075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:32.344075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:32.344075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 628/1242] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:32.346642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad5761-7a46-4241-9f1c-ecf0e585cd05 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:32.475664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:32.487957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad5761-7a46-4241-9f1c-ecf0e585cd05 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:32.535771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68418ea1-2edc-4fe1-a505-851c5b1e6ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:32.536687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68418ea1-2edc-4fe1-a505-851c5b1e6ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:32.537138 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68418ea1-2edc-4fe1-a505-851c5b1e6ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:32.537608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68418ea1-2edc-4fe1-a505-851c5b1e6ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:32.538832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68418ea1-2edc-4fe1-a505-851c5b1e6ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:32.539362 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68418ea1-2edc-4fe1-a505-851c5b1e6ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:32.541908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68418ea1-2edc-4fe1-a505-851c5b1e6ad5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:32.561128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 615/1243] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:32.568354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f390991-6f49-40bd-9766-a77a71a50467 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:32.673468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:32.701879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f390991-6f49-40bd-9766-a77a71a50467 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:32.750378 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c1f13-a8c7-4ffc-a53a-5f663f0349ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:32.751188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c1f13-a8c7-4ffc-a53a-5f663f0349ec None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:32.751562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c1f13-a8c7-4ffc-a53a-5f663f0349ec None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:32.751824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c1f13-a8c7-4ffc-a53a-5f663f0349ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:32.770233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c1f13-a8c7-4ffc-a53a-5f663f0349ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:32.771793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c1f13-a8c7-4ffc-a53a-5f663f0349ec None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-867c1f13-a8c7-4ffc-a53a-5f663f0349ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:12:32.785134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:32.786722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:32.791326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:12:32.791326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:12:32.791326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:12:32.791326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:12:32.791326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:12:32.791326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:12:32.791326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:12:32.791326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:32.791326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 629/1244] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:12:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:32.792885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68418ea1-2edc-4fe1-a505-851c5b1e6ad5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:32.953346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68418ea1-2edc-4fe1-a505-851c5b1e6ad5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:32.997813 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d780e-a1eb-4dd8-bdb0-64db56bb532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:32.998591 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d780e-a1eb-4dd8-bdb0-64db56bb532d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:32.998966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d780e-a1eb-4dd8-bdb0-64db56bb532d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:32.999374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d780e-a1eb-4dd8-bdb0-64db56bb532d None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:33.000460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d780e-a1eb-4dd8-bdb0-64db56bb532d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:33.000885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d780e-a1eb-4dd8-bdb0-64db56bb532d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:33.002745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d780e-a1eb-4dd8-bdb0-64db56bb532d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:33.018648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 616/1245] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:12:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:33.026799 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c1f13-a8c7-4ffc-a53a-5f663f0349ec None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:33.190000 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c1f13-a8c7-4ffc-a53a-5f663f0349ec None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:33.235024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064df67-5f93-4bd5-801b-9c3ebf6659b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:33.235645 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064df67-5f93-4bd5-801b-9c3ebf6659b5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:33.235930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064df67-5f93-4bd5-801b-9c3ebf6659b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:33.236314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064df67-5f93-4bd5-801b-9c3ebf6659b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:33.236950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064df67-5f93-4bd5-801b-9c3ebf6659b5 None admin] RBAC: Adding query filter params (role.id=5f8e9da552194885bbd7684f1a9c4f2d, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=55642294e8ff4f779510f4646e0e30a8) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:33.237266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064df67-5f93-4bd5-801b-9c3ebf6659b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:33.238772 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064df67-5f93-4bd5-801b-9c3ebf6659b5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:33.245850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 630/1246] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:12:33 2022] GET /identity/v3/role_assignments?user.id=55642294e8ff4f779510f4646e0e30a8&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=5f8e9da552194885bbd7684f1a9c4f2d => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:35.837833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28442240-7fc3-470b-b120-3afa56f9c535 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:35.838578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28442240-7fc3-470b-b120-3afa56f9c535 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:35.838578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28442240-7fc3-470b-b120-3afa56f9c535 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:35.839760 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 617/1247] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:12:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:12:35.846552 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed6a51-1383-4f49-b32e-ef2a67c7f989 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:35.847076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed6a51-1383-4f49-b32e-ef2a67c7f989 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:35.847614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed6a51-1383-4f49-b32e-ef2a67c7f989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:35.858538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:35.879651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-87ed6a51-1383-4f49-b32e-ef2a67c7f989 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:12:35.881032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-87ed6a51-1383-4f49-b32e-ef2a67c7f989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:12:35.977328 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 631/1248] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:12:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:12:36.270792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28442240-7fc3-470b-b120-3afa56f9c535 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:36.298066 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:36.390746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28442240-7fc3-470b-b120-3afa56f9c535 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:36.433804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59f9a7-d504-4942-ab97-306fad97f27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:36.434497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59f9a7-d504-4942-ab97-306fad97f27f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:36.434782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59f9a7-d504-4942-ab97-306fad97f27f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:36.435054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59f9a7-d504-4942-ab97-306fad97f27f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:36.447171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59f9a7-d504-4942-ab97-306fad97f27f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-2b59f9a7-d504-4942-ab97-306fad97f27f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:12:36.452125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:36.453417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:36.454535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 618/1249] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:12:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:36.459856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed6a51-1383-4f49-b32e-ef2a67c7f989 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:36.591051 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed6a51-1383-4f49-b32e-ef2a67c7f989 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:36.635682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:36.643227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb2218-e8d1-4f4e-9601-b3fff13d556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:36.644499 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb2218-e8d1-4f4e-9601-b3fff13d556f None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:36.644972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb2218-e8d1-4f4e-9601-b3fff13d556f None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:36.645177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb2218-e8d1-4f4e-9601-b3fff13d556f None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:36.645949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb2218-e8d1-4f4e-9601-b3fff13d556f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:36.646499 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb2218-e8d1-4f4e-9601-b3fff13d556f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:36.648453 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb2218-e8d1-4f4e-9601-b3fff13d556f None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:36.659893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 632/1250] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:12:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:36.667519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59f9a7-d504-4942-ab97-306fad97f27f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:36.763073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:36.790898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59f9a7-d504-4942-ab97-306fad97f27f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:36.837743 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9669c67-9ad2-4da8-992a-eacc71821fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:36.838441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9669c67-9ad2-4da8-992a-eacc71821fbc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:36.839431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9669c67-9ad2-4da8-992a-eacc71821fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:36.839431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9669c67-9ad2-4da8-992a-eacc71821fbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:36.853050 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9669c67-9ad2-4da8-992a-eacc71821fbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:36.854578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9669c67-9ad2-4da8-992a-eacc71821fbc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-f9669c67-9ad2-4da8-992a-eacc71821fbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:36.879397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:36.881103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:36.882284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:36.882284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:36.882284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:36.882284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 619/1251] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 21 16:12:36.887667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb2218-e8d1-4f4e-9601-b3fff13d556f None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:37.063717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb2218-e8d1-4f4e-9601-b3fff13d556f None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:37.118996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34c45f-9470-42ea-a674-4c0ec24a02bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:37.119569 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34c45f-9470-42ea-a674-4c0ec24a02bb None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:37.119846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34c45f-9470-42ea-a674-4c0ec24a02bb None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:37.120135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34c45f-9470-42ea-a674-4c0ec24a02bb None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:37.120812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34c45f-9470-42ea-a674-4c0ec24a02bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:37.121110 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34c45f-9470-42ea-a674-4c0ec24a02bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:37.122591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34c45f-9470-42ea-a674-4c0ec24a02bb None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:37.137363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 633/1252] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:37.144053 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9669c67-9ad2-4da8-992a-eacc71821fbc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:37.298715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9669c67-9ad2-4da8-992a-eacc71821fbc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:37.332546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:37.342370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb34e3-4149-4dd3-9dff-a16ce0ab4065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:37.343014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb34e3-4149-4dd3-9dff-a16ce0ab4065 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:37.343297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb34e3-4149-4dd3-9dff-a16ce0ab4065 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:37.343606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb34e3-4149-4dd3-9dff-a16ce0ab4065 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:37.353066 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb34e3-4149-4dd3-9dff-a16ce0ab4065 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:37.354816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb34e3-4149-4dd3-9dff-a16ce0ab4065 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-dcfb34e3-4149-4dd3-9dff-a16ce0ab4065 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:12:37.367959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:37.369182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:37.370434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:37.370434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:37.370434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:12:37.370434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:12:37.370434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:12:37.370434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:12:37.370434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:12:37.370434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:12:37.370434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 16:12:37.370434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:37.370434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 620/1253] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:37.375432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34c45f-9470-42ea-a674-4c0ec24a02bb None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:37.456903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:37.531712 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34c45f-9470-42ea-a674-4c0ec24a02bb None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:37.569989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5d567-0133-4b14-853b-d97813f16c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:37.570699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5d567-0133-4b14-853b-d97813f16c35 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:37.571007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5d567-0133-4b14-853b-d97813f16c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:37.571284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5d567-0133-4b14-853b-d97813f16c35 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:37.572079 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5d567-0133-4b14-853b-d97813f16c35 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:37.572389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5d567-0133-4b14-853b-d97813f16c35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:37.573882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5d567-0133-4b14-853b-d97813f16c35 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:37.593926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 634/1254] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Sep 21 16:12:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:37.600377 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb34e3-4149-4dd3-9dff-a16ce0ab4065 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:37.734260 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb34e3-4149-4dd3-9dff-a16ce0ab4065 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:37.768848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37d382-92d6-4bcc-b43b-8f1fa8373e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:37.769453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37d382-92d6-4bcc-b43b-8f1fa8373e88 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:37.769731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37d382-92d6-4bcc-b43b-8f1fa8373e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:37.769997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc37d382-92d6-4bcc-b43b-8f1fa8373e88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:37.828027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:37.847109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37d382-92d6-4bcc-b43b-8f1fa8373e88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:37.849413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc37d382-92d6-4bcc-b43b-8f1fa8373e88 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:37.993515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 621/1255] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:12:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:12:38.000492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5d567-0133-4b14-853b-d97813f16c35 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:38.151461 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5d567-0133-4b14-853b-d97813f16c35 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:38.166189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:38.189253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a0212-f90a-446d-852d-338061fbe757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:38.190185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a0212-f90a-446d-852d-338061fbe757 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:38.190539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a0212-f90a-446d-852d-338061fbe757 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:38.190814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a0212-f90a-446d-852d-338061fbe757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:38.202192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a0212-f90a-446d-852d-338061fbe757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:38.203639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a0212-f90a-446d-852d-338061fbe757 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-816a0212-f90a-446d-852d-338061fbe757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:38.219492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:38.220671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:38.221742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:38.221742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:38.221742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:38.221742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 635/1256] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:38.227304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37d382-92d6-4bcc-b43b-8f1fa8373e88 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:38.333860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:38.352544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc37d382-92d6-4bcc-b43b-8f1fa8373e88 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:38.388623 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091a32f-154a-44d0-a5c6-eb40a2a6316b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:38.389231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091a32f-154a-44d0-a5c6-eb40a2a6316b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:38.389509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091a32f-154a-44d0-a5c6-eb40a2a6316b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:38.389789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091a32f-154a-44d0-a5c6-eb40a2a6316b None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:38.390465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091a32f-154a-44d0-a5c6-eb40a2a6316b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:38.390772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091a32f-154a-44d0-a5c6-eb40a2a6316b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:38.392135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091a32f-154a-44d0-a5c6-eb40a2a6316b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:38.405011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 622/1257] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:38.411617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a0212-f90a-446d-852d-338061fbe757 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:38.528680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a0212-f90a-446d-852d-338061fbe757 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:38.567207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a036a6d-4f4f-45da-b99a-b6cfb957f40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:38.567915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a036a6d-4f4f-45da-b99a-b6cfb957f40c None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:38.568198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a036a6d-4f4f-45da-b99a-b6cfb957f40c None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:38.568473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a036a6d-4f4f-45da-b99a-b6cfb957f40c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:38.576401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a036a6d-4f4f-45da-b99a-b6cfb957f40c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:38.577995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a036a6d-4f4f-45da-b99a-b6cfb957f40c None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-1a036a6d-4f4f-45da-b99a-b6cfb957f40c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:12:38.592863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:38.593952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:38.594904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:12:38.594904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:12:38.594904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:12:38.594904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:12:38.594904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:12:38.594904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:12:38.594904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:12:38.594904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:38.594904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 636/1258] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:12:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:38.600114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091a32f-154a-44d0-a5c6-eb40a2a6316b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:38.717363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091a32f-154a-44d0-a5c6-eb40a2a6316b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:38.753994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ea54a-e17e-47eb-9ac9-69d8f662e868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:38.754638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ea54a-e17e-47eb-9ac9-69d8f662e868 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:38.754933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ea54a-e17e-47eb-9ac9-69d8f662e868 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:38.755183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ea54a-e17e-47eb-9ac9-69d8f662e868 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:38.755896 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ea54a-e17e-47eb-9ac9-69d8f662e868 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:38.756195 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ea54a-e17e-47eb-9ac9-69d8f662e868 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:38.757663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ea54a-e17e-47eb-9ac9-69d8f662e868 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:38.772809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 623/1259] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:38.779286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a036a6d-4f4f-45da-b99a-b6cfb957f40c None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:38.808781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:38.899403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a036a6d-4f4f-45da-b99a-b6cfb957f40c None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:38.943814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7acb47-be0f-4ef7-a380-273670ac84c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:38.944422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7acb47-be0f-4ef7-a380-273670ac84c1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:38.944699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7acb47-be0f-4ef7-a380-273670ac84c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:38.944967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7acb47-be0f-4ef7-a380-273670ac84c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:38.946424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7acb47-be0f-4ef7-a380-273670ac84c1 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=55642294e8ff4f779510f4646e0e30a8) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:38.946733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7acb47-be0f-4ef7-a380-273670ac84c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:38.948197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7acb47-be0f-4ef7-a380-273670ac84c1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:38.954342 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 637/1260] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:12:38 2022] GET /identity/v3/role_assignments?user.id=55642294e8ff4f779510f4646e0e30a8&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=459d01a218314543a7a8329e88b78331 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:41.156574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4292fdd-6a16-48fe-94e3-7e22ec66c777 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:41.157247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4292fdd-6a16-48fe-94e3-7e22ec66c777 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:41.157247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4292fdd-6a16-48fe-94e3-7e22ec66c777 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:41.158514 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 624/1261] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:12:41.163658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f85d6b-eafe-4577-9d05-c55e03905e57 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:41.164730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f85d6b-eafe-4577-9d05-c55e03905e57 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:41.164835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f85d6b-eafe-4577-9d05-c55e03905e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:41.182542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f9f85d6b-eafe-4577-9d05-c55e03905e57 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:12:41.183711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f9f85d6b-eafe-4577-9d05-c55e03905e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:12:41.249008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 638/1262] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:12:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:12:41.513638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4292fdd-6a16-48fe-94e3-7e22ec66c777 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:41.524511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:41.619847 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4292fdd-6a16-48fe-94e3-7e22ec66c777 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:41.656558 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4336538-440c-4bc9-96d2-0f06f8abdbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:41.657183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4336538-440c-4bc9-96d2-0f06f8abdbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:41.657460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4336538-440c-4bc9-96d2-0f06f8abdbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:41.657726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4336538-440c-4bc9-96d2-0f06f8abdbf2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:41.668549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4336538-440c-4bc9-96d2-0f06f8abdbf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c4336538-440c-4bc9-96d2-0f06f8abdbf2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:12:41.673299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:41.674499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:41.675400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 625/1263] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:12:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:41.680450 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f85d6b-eafe-4577-9d05-c55e03905e57 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:41.698269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:41.778371 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f85d6b-eafe-4577-9d05-c55e03905e57 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:41.811535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9739d0c-f565-405b-ba04-a777af9228a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:41.812133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9739d0c-f565-405b-ba04-a777af9228a4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:41.812429 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9739d0c-f565-405b-ba04-a777af9228a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:41.812678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9739d0c-f565-405b-ba04-a777af9228a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:41.813333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9739d0c-f565-405b-ba04-a777af9228a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:41.813634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9739d0c-f565-405b-ba04-a777af9228a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:41.815263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9739d0c-f565-405b-ba04-a777af9228a4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:41.823223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 639/1264] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:12:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:41.828972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4336538-440c-4bc9-96d2-0f06f8abdbf2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:41.908453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:41.941508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4336538-440c-4bc9-96d2-0f06f8abdbf2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:41.974474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08397b9-ea4e-4584-9396-f828fe23e705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:41.975087 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08397b9-ea4e-4584-9396-f828fe23e705 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:41.975361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08397b9-ea4e-4584-9396-f828fe23e705 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:41.975629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08397b9-ea4e-4584-9396-f828fe23e705 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:41.985273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08397b9-ea4e-4584-9396-f828fe23e705 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:41.986732 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08397b9-ea4e-4584-9396-f828fe23e705 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a08397b9-ea4e-4584-9396-f828fe23e705 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:42.000735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:42.001830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:42.002799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:42.002799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:42.002799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:42.002799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 626/1265] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:42.008104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9739d0c-f565-405b-ba04-a777af9228a4 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:42.096740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:42.115500 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9739d0c-f565-405b-ba04-a777af9228a4 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:42.152770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723754c0-d719-4861-81ea-89bc37990d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:42.153798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723754c0-d719-4861-81ea-89bc37990d5b None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:42.154332 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723754c0-d719-4861-81ea-89bc37990d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:42.154859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723754c0-d719-4861-81ea-89bc37990d5b None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:42.155827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723754c0-d719-4861-81ea-89bc37990d5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:42.156382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723754c0-d719-4861-81ea-89bc37990d5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:42.158846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723754c0-d719-4861-81ea-89bc37990d5b None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:42.179075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 640/1266] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:42.184890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08397b9-ea4e-4584-9396-f828fe23e705 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:42.288274 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08397b9-ea4e-4584-9396-f828fe23e705 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:42.308308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:42.323327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d7571-89e4-45eb-9612-d339811dba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:42.323948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d7571-89e4-45eb-9612-d339811dba91 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:42.324229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d7571-89e4-45eb-9612-d339811dba91 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:42.324501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d7571-89e4-45eb-9612-d339811dba91 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:42.332003 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d7571-89e4-45eb-9612-d339811dba91 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:42.333434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d7571-89e4-45eb-9612-d339811dba91 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-934d7571-89e4-45eb-9612-d339811dba91 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:12:42.345078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:42.346244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:42.347718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:42.347718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:42.347718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:12:42.347718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:12:42.347718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:12:42.347718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:12:42.347718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:12:42.347718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:12:42.347718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 16:12:42.347718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:42.347718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 627/1267] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:42.352108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723754c0-d719-4861-81ea-89bc37990d5b None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:42.450152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723754c0-d719-4861-81ea-89bc37990d5b None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:42.483830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413b1b8-f766-43f7-be0f-ead49acbb470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:42.484437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413b1b8-f766-43f7-be0f-ead49acbb470 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:42.484783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413b1b8-f766-43f7-be0f-ead49acbb470 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:42.485022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413b1b8-f766-43f7-be0f-ead49acbb470 None admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:42.485678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413b1b8-f766-43f7-be0f-ead49acbb470 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:42.485975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413b1b8-f766-43f7-be0f-ead49acbb470 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:42.487401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413b1b8-f766-43f7-be0f-ead49acbb470 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:42.496949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:42.506533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 641/1268] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Sep 21 16:12:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:42.513009 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d7571-89e4-45eb-9612-d339811dba91 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:42.613338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d7571-89e4-45eb-9612-d339811dba91 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:42.650394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19025f17-06dd-4919-82f8-48bc5827c3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:42.651017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19025f17-06dd-4919-82f8-48bc5827c3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:42.651295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19025f17-06dd-4919-82f8-48bc5827c3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:42.651570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19025f17-06dd-4919-82f8-48bc5827c3d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:42.662385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19025f17-06dd-4919-82f8-48bc5827c3d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:42.663844 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19025f17-06dd-4919-82f8-48bc5827c3d4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-19025f17-06dd-4919-82f8-48bc5827c3d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:42.677968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:42.679185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:42.680858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:42.680858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:42.680858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:42.680858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 628/1269] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:42.685723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413b1b8-f766-43f7-be0f-ead49acbb470 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:42.784504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413b1b8-f766-43f7-be0f-ead49acbb470 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:42.819160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160e094-7cb8-4a88-b30d-1a153f9ec6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:42.819754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160e094-7cb8-4a88-b30d-1a153f9ec6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:42.820032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160e094-7cb8-4a88-b30d-1a153f9ec6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:42.820300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160e094-7cb8-4a88-b30d-1a153f9ec6a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:42.820930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160e094-7cb8-4a88-b30d-1a153f9ec6a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:42.821230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160e094-7cb8-4a88-b30d-1a153f9ec6a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:42.822610 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160e094-7cb8-4a88-b30d-1a153f9ec6a1 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:42.833607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 642/1270] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:42.839643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19025f17-06dd-4919-82f8-48bc5827c3d4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:42.871990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:42.946339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19025f17-06dd-4919-82f8-48bc5827c3d4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:42.981112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddaac7-1fc7-42eb-8614-d26ce6b220a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:42.981742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddaac7-1fc7-42eb-8614-d26ce6b220a6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:42.982023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddaac7-1fc7-42eb-8614-d26ce6b220a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:42.982331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddaac7-1fc7-42eb-8614-d26ce6b220a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:42.991086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddaac7-1fc7-42eb-8614-d26ce6b220a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:42.993715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddaac7-1fc7-42eb-8614-d26ce6b220a6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0eddaac7-1fc7-42eb-8614-d26ce6b220a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:12:43.005882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:43.007054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:43.008295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:12:43.008295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:12:43.008295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:12:43.008295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:12:43.008295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:12:43.008295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:12:43.008295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:12:43.008295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:43.008295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 629/1271] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:43.013108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160e094-7cb8-4a88-b30d-1a153f9ec6a1 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:43.076398 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:43.119325 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160e094-7cb8-4a88-b30d-1a153f9ec6a1 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:43.152443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdba7b3-6206-4712-83b5-afc9abf74272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:43.153061 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdba7b3-6206-4712-83b5-afc9abf74272 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:43.153348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdba7b3-6206-4712-83b5-afc9abf74272 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:43.153624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdba7b3-6206-4712-83b5-afc9abf74272 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:43.154350 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdba7b3-6206-4712-83b5-afc9abf74272 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:43.154660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdba7b3-6206-4712-83b5-afc9abf74272 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:43.156078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdba7b3-6206-4712-83b5-afc9abf74272 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:43.168787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 643/1272] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:12:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:43.174771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddaac7-1fc7-42eb-8614-d26ce6b220a6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:43.265174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:43.275461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddaac7-1fc7-42eb-8614-d26ce6b220a6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:43.309804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4659c194-ff73-443f-8761-6c226f000501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:43.310485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659c194-ff73-443f-8761-6c226f000501 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:43.310766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659c194-ff73-443f-8761-6c226f000501 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:43.311044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4659c194-ff73-443f-8761-6c226f000501 None admin] PATH_INFO: `/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/55642294e8ff4f779510f4646e0e30a8/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:43.313038 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659c194-ff73-443f-8761-6c226f000501 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3c9e1d9331b4d4dbd3dd447937c42ad, user_id=55642294e8ff4f779510f4646e0e30a8, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:43.314767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4659c194-ff73-443f-8761-6c226f000501 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:43.327642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 630/1273] 10.222.0.64 () {58 vars in 1439 bytes} [Wed Sep 21 16:12:43 2022] PUT /identity/v3/projects/c3c9e1d9331b4d4dbd3dd447937c42ad/users/55642294e8ff4f779510f4646e0e30a8/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:12:45.469918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc4de0-cdc4-4fd0-a3a9-c33e574c956b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:45.470507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc4de0-cdc4-4fd0-a3a9-c33e574c956b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:45.470507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cc4de0-cdc4-4fd0-a3a9-c33e574c956b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:45.471830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 644/1274] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:12:45.477197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1fd3a-6be9-4f43-802f-dd7800f615f9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:45.477606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1fd3a-6be9-4f43-802f-dd7800f615f9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:45.477749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e1fd3a-6be9-4f43-802f-dd7800f615f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:45.496236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-83e1fd3a-6be9-4f43-802f-dd7800f615f9 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:12:45.497419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-83e1fd3a-6be9-4f43-802f-dd7800f615f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:12:45.562827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 631/1275] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:12:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:12:45.846585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc4de0-cdc4-4fd0-a3a9-c33e574c956b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:45.954740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cc4de0-cdc4-4fd0-a3a9-c33e574c956b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:45.971849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:45.988949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df2b39-930b-484b-9f79-f9e11cb19477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:45.989578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df2b39-930b-484b-9f79-f9e11cb19477 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:45.989885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df2b39-930b-484b-9f79-f9e11cb19477 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:45.990201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df2b39-930b-484b-9f79-f9e11cb19477 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:46.000605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df2b39-930b-484b-9f79-f9e11cb19477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-b4df2b39-930b-484b-9f79-f9e11cb19477 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 21 16:12:46.005454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:46.006570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:46.007634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 645/1276] 10.222.0.64 () {56 vars in 1096 bytes} [Wed Sep 21 16:12:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:46.012904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1fd3a-6be9-4f43-802f-dd7800f615f9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:46.106782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:46.127316 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e1fd3a-6be9-4f43-802f-dd7800f615f9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:46.161217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc05a7-4bc7-431f-8410-4ea71fe1b8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:46.161811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc05a7-4bc7-431f-8410-4ea71fe1b8dd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:46.162110 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc05a7-4bc7-431f-8410-4ea71fe1b8dd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:46.162387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc05a7-4bc7-431f-8410-4ea71fe1b8dd None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:46.163039 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc05a7-4bc7-431f-8410-4ea71fe1b8dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:46.163337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc05a7-4bc7-431f-8410-4ea71fe1b8dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:46.165327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc05a7-4bc7-431f-8410-4ea71fe1b8dd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:46.173051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 632/1277] 10.222.0.64 () {56 vars in 1099 bytes} [Wed Sep 21 16:12:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:46.178898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df2b39-930b-484b-9f79-f9e11cb19477 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:46.277635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df2b39-930b-484b-9f79-f9e11cb19477 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:46.310589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf68aa-038b-4213-b454-cfdfd573f0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:46.311211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf68aa-038b-4213-b454-cfdfd573f0ac None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:46.311487 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf68aa-038b-4213-b454-cfdfd573f0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:46.311756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf68aa-038b-4213-b454-cfdfd573f0ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:46.321556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf68aa-038b-4213-b454-cfdfd573f0ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:46.323015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf68aa-038b-4213-b454-cfdfd573f0ac None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-e8cf68aa-038b-4213-b454-cfdfd573f0ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:46.337514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:46.338701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:46.340037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:46.340037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:46.340037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:46.340037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 646/1278] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:46.344432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc05a7-4bc7-431f-8410-4ea71fe1b8dd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:46.444138 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc05a7-4bc7-431f-8410-4ea71fe1b8dd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:46.478634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b5ef9-bd64-414f-a30b-8d13d0dfcff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:46.479290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b5ef9-bd64-414f-a30b-8d13d0dfcff5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:46.479492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b5ef9-bd64-414f-a30b-8d13d0dfcff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:46.479761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b5ef9-bd64-414f-a30b-8d13d0dfcff5 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:46.480431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b5ef9-bd64-414f-a30b-8d13d0dfcff5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:46.480732 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b5ef9-bd64-414f-a30b-8d13d0dfcff5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:46.482099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b5ef9-bd64-414f-a30b-8d13d0dfcff5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:46.494582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 633/1279] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:46.500425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf68aa-038b-4213-b454-cfdfd573f0ac None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:46.534216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:46.606156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf68aa-038b-4213-b454-cfdfd573f0ac None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:46.639043 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66cc92-c666-4985-a275-595bd58bd7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:46.639670 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66cc92-c666-4985-a275-595bd58bd7ee None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:46.639950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66cc92-c666-4985-a275-595bd58bd7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:46.640222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e66cc92-c666-4985-a275-595bd58bd7ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:46.647786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66cc92-c666-4985-a275-595bd58bd7ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:46.649786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e66cc92-c666-4985-a275-595bd58bd7ee None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-0e66cc92-c666-4985-a275-595bd58bd7ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 21 16:12:46.661416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:46.662403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:46.663376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:46.663376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:46.663376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 21 16:12:46.663376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 21 16:12:46.663376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 21 16:12:46.663376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 21 16:12:46.663376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 21 16:12:46.663376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 21 16:12:46.663376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 21 16:12:46.663376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:46.663376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 647/1280] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:46.668806 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b5ef9-bd64-414f-a30b-8d13d0dfcff5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:46.672620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:46.769640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b5ef9-bd64-414f-a30b-8d13d0dfcff5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:46.803604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f62d31-7cf5-4208-b1b8-b992eb4db0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:46.804284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f62d31-7cf5-4208-b1b8-b992eb4db0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:46.804607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f62d31-7cf5-4208-b1b8-b992eb4db0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:46.804924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f62d31-7cf5-4208-b1b8-b992eb4db0b6 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:46.805699 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f62d31-7cf5-4208-b1b8-b992eb4db0b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:46.806075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f62d31-7cf5-4208-b1b8-b992eb4db0b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:46.807554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f62d31-7cf5-4208-b1b8-b992eb4db0b6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:46.826323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 634/1281] 10.222.0.64 () {56 vars in 1143 bytes} [Wed Sep 21 16:12:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:46.832265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66cc92-c666-4985-a275-595bd58bd7ee None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:46.927489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:46.939595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e66cc92-c666-4985-a275-595bd58bd7ee None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:46.973880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4be268-e80e-4097-8570-a3c03ba03e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:46.974509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be268-e80e-4097-8570-a3c03ba03e29 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:46.974783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be268-e80e-4097-8570-a3c03ba03e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:46.975056 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4be268-e80e-4097-8570-a3c03ba03e29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:47.044327 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4be268-e80e-4097-8570-a3c03ba03e29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:47.045680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4be268-e80e-4097-8570-a3c03ba03e29 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:47.132811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:47.143085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 648/1282] 10.222.0.64 () {58 vars in 1282 bytes} [Wed Sep 21 16:12:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 21 16:12:47.149449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f62d31-7cf5-4208-b1b8-b992eb4db0b6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:47.210401 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:47.250710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f62d31-7cf5-4208-b1b8-b992eb4db0b6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:47.284400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e172c3-3d90-48fc-a9fb-78c49b18c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:47.285020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e172c3-3d90-48fc-a9fb-78c49b18c382 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:47.285295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e172c3-3d90-48fc-a9fb-78c49b18c382 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:47.285563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e172c3-3d90-48fc-a9fb-78c49b18c382 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:47.295447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e172c3-3d90-48fc-a9fb-78c49b18c382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:47.296812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e172c3-3d90-48fc-a9fb-78c49b18c382 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-74e172c3-3d90-48fc-a9fb-78c49b18c382 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 21 16:12:47.311022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:47.312135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 21 16:12:47.313461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 21 16:12:47.313461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 21 16:12:47.313461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:47.313461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 635/1283] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:47.318324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4be268-e80e-4097-8570-a3c03ba03e29 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:47.417037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4be268-e80e-4097-8570-a3c03ba03e29 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:47.451673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a4db4-80ec-435e-ad57-e68d0cba67ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:47.452276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a4db4-80ec-435e-ad57-e68d0cba67ba None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:47.452555 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a4db4-80ec-435e-ad57-e68d0cba67ba None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:47.452826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6a4db4-80ec-435e-ad57-e68d0cba67ba None admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:47.453468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a4db4-80ec-435e-ad57-e68d0cba67ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:47.453776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a4db4-80ec-435e-ad57-e68d0cba67ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:47.455148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6a4db4-80ec-435e-ad57-e68d0cba67ba None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:47.467256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 649/1284] 10.222.0.64 () {56 vars in 1109 bytes} [Wed Sep 21 16:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:47.473141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e172c3-3d90-48fc-a9fb-78c49b18c382 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:47.575059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e172c3-3d90-48fc-a9fb-78c49b18c382 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:47.588992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:47.609767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef556d-d842-4e60-8e95-e46ae9c1e4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:47.610420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef556d-d842-4e60-8e95-e46ae9c1e4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:47.610699 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef556d-d842-4e60-8e95-e46ae9c1e4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:47.610966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef556d-d842-4e60-8e95-e46ae9c1e4d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:47.618183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef556d-d842-4e60-8e95-e46ae9c1e4d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:47.619593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef556d-d842-4e60-8e95-e46ae9c1e4d1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-03ef556d-d842-4e60-8e95-e46ae9c1e4d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 21 16:12:47.630605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:47.631657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:47.633562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 16:12:47.633562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 16:12:47.633562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 16:12:47.633562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 16:12:47.633562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 16:12:47.633562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 16:12:47.633562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 21 16:12:47.633562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 16:12:47.633562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 636/1285] 10.222.0.64 () {56 vars in 1111 bytes} [Wed Sep 21 16:12:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:47.638172 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a4db4-80ec-435e-ad57-e68d0cba67ba None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:47.690899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:47.740654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6a4db4-80ec-435e-ad57-e68d0cba67ba None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:47.773449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd397d-e836-4bf3-83c1-52aab36bcd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:47.774061 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd397d-e836-4bf3-83c1-52aab36bcd20 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:47.774368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd397d-e836-4bf3-83c1-52aab36bcd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:47.774639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd397d-e836-4bf3-83c1-52aab36bcd20 None admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:47.775328 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd397d-e836-4bf3-83c1-52aab36bcd20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:47.775630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd397d-e836-4bf3-83c1-52aab36bcd20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:47.777035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd397d-e836-4bf3-83c1-52aab36bcd20 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:47.790157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 650/1286] 10.222.0.64 () {56 vars in 1148 bytes} [Wed Sep 21 16:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:47.796562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef556d-d842-4e60-8e95-e46ae9c1e4d1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:47.897332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef556d-d842-4e60-8e95-e46ae9c1e4d1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:47.931026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acd793-3811-496c-8023-f04752941360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:47.931625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acd793-3811-496c-8023-f04752941360 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:47.931902 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acd793-3811-496c-8023-f04752941360 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:47.932173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acd793-3811-496c-8023-f04752941360 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:47.932890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acd793-3811-496c-8023-f04752941360 None admin] RBAC: Adding query filter params (role.id=459d01a218314543a7a8329e88b78331, scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad, user.id=55642294e8ff4f779510f4646e0e30a8) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:47.933197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acd793-3811-496c-8023-f04752941360 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:47.934641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0acd793-3811-496c-8023-f04752941360 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:47.940654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 637/1287] 10.222.0.64 () {56 vars in 1374 bytes} [Wed Sep 21 16:12:47 2022] GET /identity/v3/role_assignments?user.id=55642294e8ff4f779510f4646e0e30a8&scope.project.id=c3c9e1d9331b4d4dbd3dd447937c42ad&role.id=459d01a218314543a7a8329e88b78331 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:50.154586 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e993d-3635-4ed0-8a05-068ae679c99c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:50.155155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e993d-3635-4ed0-8a05-068ae679c99c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:50.155155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9e993d-3635-4ed0-8a05-068ae679c99c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:50.156470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 651/1288] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:12:50.161673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2231451-7e17-40cb-bcab-81607b04942f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:50.162090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2231451-7e17-40cb-bcab-81607b04942f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:50.162264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2231451-7e17-40cb-bcab-81607b04942f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:50.181362 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d2231451-7e17-40cb-bcab-81607b04942f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:12:50.182578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d2231451-7e17-40cb-bcab-81607b04942f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:12:50.211868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:50.248966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 638/1289] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:12:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:12:50.518204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e993d-3635-4ed0-8a05-068ae679c99c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:50.626613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9e993d-3635-4ed0-8a05-068ae679c99c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:50.638075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:50.663066 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544410e-874a-4c49-93df-04aa1f9c181e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:50.664303 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544410e-874a-4c49-93df-04aa1f9c181e None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:50.664584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544410e-874a-4c49-93df-04aa1f9c181e None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:50.664855 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544410e-874a-4c49-93df-04aa1f9c181e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:50.665440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544410e-874a-4c49-93df-04aa1f9c181e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:50.666913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6544410e-874a-4c49-93df-04aa1f9c181e None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-6544410e-874a-4c49-93df-04aa1f9c181e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:50.675675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:50.676913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:50.678070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 652/1290] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Sep 21 16:12:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:50.682875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2231451-7e17-40cb-bcab-81607b04942f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:50.780363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2231451-7e17-40cb-bcab-81607b04942f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:50.813177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d7d3e-d4e8-4d79-8c82-8e9d46858221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:50.813773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d7d3e-d4e8-4d79-8c82-8e9d46858221 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:50.814339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d7d3e-d4e8-4d79-8c82-8e9d46858221 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:50.814612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d7d3e-d4e8-4d79-8c82-8e9d46858221 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:50.815238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d7d3e-d4e8-4d79-8c82-8e9d46858221 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:50.815537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d7d3e-d4e8-4d79-8c82-8e9d46858221 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:50.816884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40d7d3e-d4e8-4d79-8c82-8e9d46858221 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:50.821573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 639/1291] 10.222.0.64 () {56 vars in 1116 bytes} [Wed Sep 21 16:12:50 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:50.827357 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544410e-874a-4c49-93df-04aa1f9c181e None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:50.929111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6544410e-874a-4c49-93df-04aa1f9c181e None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:50.965370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d12f8-493b-445c-a1b1-c9be6a172916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:50.965976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d12f8-493b-445c-a1b1-c9be6a172916 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:50.966289 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d12f8-493b-445c-a1b1-c9be6a172916 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:50.966564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d12f8-493b-445c-a1b1-c9be6a172916 None admin] PATH_INFO: `/v3/services` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:50.967197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d12f8-493b-445c-a1b1-c9be6a172916 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:50.967503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d12f8-493b-445c-a1b1-c9be6a172916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:50.968854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d12f8-493b-445c-a1b1-c9be6a172916 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:50.973292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 653/1292] 10.222.0.64 () {56 vars in 1116 bytes} [Wed Sep 21 16:12:50 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:53.128156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab2718-5356-4cf3-9f1c-f37cb94471e6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:53.128717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab2718-5356-4cf3-9f1c-f37cb94471e6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:53.128717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab2718-5356-4cf3-9f1c-f37cb94471e6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:53.130008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 640/1293] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:12:53.135310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae646e1e-95f5-4ea1-a960-d02504511a16 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:53.135765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae646e1e-95f5-4ea1-a960-d02504511a16 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:53.135859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae646e1e-95f5-4ea1-a960-d02504511a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:53.154635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ae646e1e-95f5-4ea1-a960-d02504511a16 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:12:53.155834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ae646e1e-95f5-4ea1-a960-d02504511a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:12:53.184827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:53.222881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 654/1294] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:12:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:12:53.487521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab2718-5356-4cf3-9f1c-f37cb94471e6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:53.520343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:53.589910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab2718-5356-4cf3-9f1c-f37cb94471e6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:53.624178 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e9e06-bbce-4cf2-80c8-4dc0258ed7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:53.624780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e9e06-bbce-4cf2-80c8-4dc0258ed7b0 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:53.625055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e9e06-bbce-4cf2-80c8-4dc0258ed7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:53.625323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e9e06-bbce-4cf2-80c8-4dc0258ed7b0 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:53.625966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e9e06-bbce-4cf2-80c8-4dc0258ed7b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:53.627347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e9e06-bbce-4cf2-80c8-4dc0258ed7b0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:53.645365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 641/1295] 10.222.0.64 () {60 vars in 1141 bytes} [Wed Sep 21 16:12:53 2022] POST /identity/v3/services => generated 246 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:12:55.850740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa41e6b-9fab-4fce-a59e-f733550bc0ae None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:55.851259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa41e6b-9fab-4fce-a59e-f733550bc0ae None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:55.851259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa41e6b-9fab-4fce-a59e-f733550bc0ae None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:55.852537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 655/1296] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:12:55.857857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b8910-9db8-45b6-b3c4-efd632712648 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:55.858306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b8910-9db8-45b6-b3c4-efd632712648 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:55.858395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b8910-9db8-45b6-b3c4-efd632712648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:55.878133 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9a1b8910-9db8-45b6-b3c4-efd632712648 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:12:55.879340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9a1b8910-9db8-45b6-b3c4-efd632712648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:12:55.951749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 642/1297] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:12:55 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:12:56.235053 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa41e6b-9fab-4fce-a59e-f733550bc0ae None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:56.334482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa41e6b-9fab-4fce-a59e-f733550bc0ae None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:56.369891 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef35145-e652-43f6-837f-0d664fd77fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:56.370551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef35145-e652-43f6-837f-0d664fd77fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:56.370828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef35145-e652-43f6-837f-0d664fd77fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:56.371099 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef35145-e652-43f6-837f-0d664fd77fb5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:56.371677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef35145-e652-43f6-837f-0d664fd77fb5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:56.373023 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef35145-e652-43f6-837f-0d664fd77fb5 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-1ef35145-e652-43f6-837f-0d664fd77fb5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:56.381653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:56.382843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:56.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 656/1298] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Sep 21 16:12:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:56.388941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b8910-9db8-45b6-b3c4-efd632712648 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:56.407191 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:56.489484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b8910-9db8-45b6-b3c4-efd632712648 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:56.522529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe7872-a895-4517-8414-551e1e87a7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:56.523048 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe7872-a895-4517-8414-551e1e87a7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:56.523325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe7872-a895-4517-8414-551e1e87a7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:56.523619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe7872-a895-4517-8414-551e1e87a7a1 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:56.524240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe7872-a895-4517-8414-551e1e87a7a1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:56.524545 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe7872-a895-4517-8414-551e1e87a7a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:56.525884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fe7872-a895-4517-8414-551e1e87a7a1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:56.530851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 643/1299] 10.222.0.64 () {56 vars in 1116 bytes} [Wed Sep 21 16:12:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:56.536829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef35145-e652-43f6-837f-0d664fd77fb5 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:56.568590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:56.635311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef35145-e652-43f6-837f-0d664fd77fb5 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:56.667992 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39518cfc-1891-490e-bcda-5517646427d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:56.668582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39518cfc-1891-490e-bcda-5517646427d4 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:56.668858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39518cfc-1891-490e-bcda-5517646427d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:56.669126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39518cfc-1891-490e-bcda-5517646427d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:56.669816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39518cfc-1891-490e-bcda-5517646427d4 None admin] RBAC: Adding query filter params (interface=public, service_id=8546047b33a14d7b9084a2737e284090, region_id=RegionOne) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:56.670150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39518cfc-1891-490e-bcda-5517646427d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:56.671490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39518cfc-1891-490e-bcda-5517646427d4 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:56.676369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 657/1300] 10.222.0.64 () {56 vars in 1251 bytes} [Wed Sep 21 16:12:56 2022] GET /identity/v3/endpoints?service_id=8546047b33a14d7b9084a2737e284090&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:12:56.682064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe7872-a895-4517-8414-551e1e87a7a1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:56.762979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:56.781550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe7872-a895-4517-8414-551e1e87a7a1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:56.815721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39315224-9d83-40cc-b5eb-0a209b5ce73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:56.816311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39315224-9d83-40cc-b5eb-0a209b5ce73d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:56.816582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39315224-9d83-40cc-b5eb-0a209b5ce73d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:56.816852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39315224-9d83-40cc-b5eb-0a209b5ce73d None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:56.817443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39315224-9d83-40cc-b5eb-0a209b5ce73d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:56.818961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39315224-9d83-40cc-b5eb-0a209b5ce73d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:56.825669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 644/1301] 10.222.0.64 () {56 vars in 1087 bytes} [Wed Sep 21 16:12:56 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:12:59.004728 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f70fc-597b-409b-9571-c7b0d957d727 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:59.005258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f70fc-597b-409b-9571-c7b0d957d727 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:59.005258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f70fc-597b-409b-9571-c7b0d957d727 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:59.006716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 658/1302] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:12:59.011935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bd6b7-91f1-4276-931e-0a0b36533363 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:59.012341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bd6b7-91f1-4276-931e-0a0b36533363 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:59.012472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bd6b7-91f1-4276-931e-0a0b36533363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:59.030807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-408bd6b7-91f1-4276-931e-0a0b36533363 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:12:59.031987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-408bd6b7-91f1-4276-931e-0a0b36533363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:12:59.096571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 645/1303] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:12:59 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 21 16:12:59.364941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f70fc-597b-409b-9571-c7b0d957d727 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:59.461803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:59.468542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f70fc-597b-409b-9571-c7b0d957d727 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:59.501690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58442418-e35a-47c3-b8db-12fb0819ccd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:59.502351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58442418-e35a-47c3-b8db-12fb0819ccd3 None admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:59.502629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58442418-e35a-47c3-b8db-12fb0819ccd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:59.502899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58442418-e35a-47c3-b8db-12fb0819ccd3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:59.503480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58442418-e35a-47c3-b8db-12fb0819ccd3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:59.504822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58442418-e35a-47c3-b8db-12fb0819ccd3 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application [None req-58442418-e35a-47c3-b8db-12fb0819ccd3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 16:12:59.514300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 16:12:59.518242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application with Lock( Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self._enter() Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application return self.creator() Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application created_value = creator( Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: ERROR keystone.server.flask.application Sep 21 16:12:59.519355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 659/1304] 10.222.0.64 () {56 vars in 1117 bytes} [Wed Sep 21 16:12:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 21 16:12:59.521367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bd6b7-91f1-4276-931e-0a0b36533363 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:59.596858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:12:59.619743 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bd6b7-91f1-4276-931e-0a0b36533363 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:59.652587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527c8823-e350-423a-bf9e-43194070c5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:59.653178 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c8823-e350-423a-bf9e-43194070c5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:59.653458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c8823-e350-423a-bf9e-43194070c5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:59.653730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c8823-e350-423a-bf9e-43194070c5a8 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:59.654385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c8823-e350-423a-bf9e-43194070c5a8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:12:59.654688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c8823-e350-423a-bf9e-43194070c5a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:59.656044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527c8823-e350-423a-bf9e-43194070c5a8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:59.660990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 646/1305] 10.222.0.64 () {56 vars in 1116 bytes} [Wed Sep 21 16:12:59 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:12:59.666826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58442418-e35a-47c3-b8db-12fb0819ccd3 None admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:12:59.772861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58442418-e35a-47c3-b8db-12fb0819ccd3 None admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:12:59.805279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9946e3-db8d-4233-b844-dffb6844f1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:12:59.805874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9946e3-db8d-4233-b844-dffb6844f1d9 None admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:12:59.806217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9946e3-db8d-4233-b844-dffb6844f1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:12:59.806491 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9946e3-db8d-4233-b844-dffb6844f1d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:12:59.807157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9946e3-db8d-4233-b844-dffb6844f1d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:12:59.808496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9946e3-db8d-4233-b844-dffb6844f1d9 None admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:12:59.829053 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 660/1306] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:12:59 2022] POST /identity/v3/endpoints => generated 331 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:13:09.658724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=30 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:13:09.824820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:08.945835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fc77b-7f5b-4bd6-a152-1df05e0f39fe None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:08.947484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fc77b-7f5b-4bd6-a152-1df05e0f39fe None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:08.948408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fc77b-7f5b-4bd6-a152-1df05e0f39fe None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:08.958970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 647/1307] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:08 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:08.959986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032c5b1-b43a-4eb7-a597-1aa4d9c4cf20 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:08.960968 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032c5b1-b43a-4eb7-a597-1aa4d9c4cf20 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:08.961466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f032c5b1-b43a-4eb7-a597-1aa4d9c4cf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:09.112934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f032c5b1-b43a-4eb7-a597-1aa4d9c4cf20 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:09.115337 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f032c5b1-b43a-4eb7-a597-1aa4d9c4cf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:09.246093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 661/1308] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:09.553659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a6cb7-93f0-4bb7-a970-c8f60b51dc46 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:09.553938 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a6cb7-93f0-4bb7-a970-c8f60b51dc46 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:09.554235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a6cb7-93f0-4bb7-a970-c8f60b51dc46 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:09.555574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 648/1309] 10.222.0.64 () {54 vars in 963 bytes} [Wed Sep 21 16:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:09.564841 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6379b-3c85-411c-ae6e-eeae589912d8 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:09.565335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6379b-3c85-411c-ae6e-eeae589912d8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:09.565828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6379b-3c85-411c-ae6e-eeae589912d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:09.681274 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-efe6379b-3c85-411c-ae6e-eeae589912d8 None None] MFA Rules not processed for user `e175b8de037d45d48cbddd64e552f6f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:09.682919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-efe6379b-3c85-411c-ae6e-eeae589912d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:09.709249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:09.793726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a6cb7-93f0-4bb7-a970-c8f60b51dc46 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:09.796205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a87a6cb7-93f0-4bb7-a970-c8f60b51dc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:09.800157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 662/1310] 10.222.0.64 () {58 vars in 1060 bytes} [Wed Sep 21 16:14:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:09.995582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a6cb7-93f0-4bb7-a970-c8f60b51dc46 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:10.094249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98b6ea-df96-4980-a0d9-024481e35e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:10.096103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98b6ea-df96-4980-a0d9-024481e35e70 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:10.096577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98b6ea-df96-4980-a0d9-024481e35e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:10.097052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98b6ea-df96-4980-a0d9-024481e35e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:10.100390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4e98b6ea-df96-4980-a0d9-024481e35e70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:10.216850 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:10.327583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98b6ea-df96-4980-a0d9-024481e35e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:10.330705 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98b6ea-df96-4980-a0d9-024481e35e70 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:10.365670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 649/1311] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:14:12.954534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929548a9-d601-46dd-bba1-82e633f79d74 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:12.955089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929548a9-d601-46dd-bba1-82e633f79d74 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:12.955089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929548a9-d601-46dd-bba1-82e633f79d74 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:12.956349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 663/1312] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:12.962716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0b1b9-c8dd-4427-9808-0b47ef4cc7fe None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:12.963255 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0b1b9-c8dd-4427-9808-0b47ef4cc7fe None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:12.963389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b0b1b9-c8dd-4427-9808-0b47ef4cc7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:13.051021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-03b0b1b9-c8dd-4427-9808-0b47ef4cc7fe None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:13.052249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-03b0b1b9-c8dd-4427-9808-0b47ef4cc7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:13.158839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 650/1313] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:13.452361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929548a9-d601-46dd-bba1-82e633f79d74 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:13.453998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-929548a9-d601-46dd-bba1-82e633f79d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:13.580330 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929548a9-d601-46dd-bba1-82e633f79d74 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:13.661539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:13.664245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76633a54-93eb-4d44-93fd-c4abe20ed5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:13.665084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76633a54-93eb-4d44-93fd-c4abe20ed5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:13.665531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76633a54-93eb-4d44-93fd-c4abe20ed5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:13.666127 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76633a54-93eb-4d44-93fd-c4abe20ed5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:13.667797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-76633a54-93eb-4d44-93fd-c4abe20ed5d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:13.784355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76633a54-93eb-4d44-93fd-c4abe20ed5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:13.785808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76633a54-93eb-4d44-93fd-c4abe20ed5d4 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:13.805284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 664/1314] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:14:16.341467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c85ac-678a-4c16-8bb4-3e7bf55c80fc None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:16.342006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c85ac-678a-4c16-8bb4-3e7bf55c80fc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:16.342006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c85ac-678a-4c16-8bb4-3e7bf55c80fc None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:16.343380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 651/1315] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:16.348697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d92e4-3883-47ff-8968-dc3a7a0659a8 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:16.349096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d92e4-3883-47ff-8968-dc3a7a0659a8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:16.349253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d92e4-3883-47ff-8968-dc3a7a0659a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:16.414357 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d27d92e4-3883-47ff-8968-dc3a7a0659a8 None None] MFA Rules not processed for user `1888f4b82c7940fdb9abfe0a57b404c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:16.415583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d27d92e4-3883-47ff-8968-dc3a7a0659a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:16.473007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:16.518390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 665/1316] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:16.820963 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73aa250-8c7a-4480-a32f-c4279c662e8c None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:16.821252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73aa250-8c7a-4480-a32f-c4279c662e8c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:16.821534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73aa250-8c7a-4480-a32f-c4279c662e8c None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:16.822905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 652/1317] 10.222.0.64 () {54 vars in 963 bytes} [Wed Sep 21 16:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:16.829498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f7b90-55d6-437d-928e-f073d0cefae3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:16.829786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f7b90-55d6-437d-928e-f073d0cefae3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:16.830069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7f7b90-55d6-437d-928e-f073d0cefae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:16.906653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4d7f7b90-55d6-437d-928e-f073d0cefae3 None None] MFA Rules not processed for user `e175b8de037d45d48cbddd64e552f6f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:16.908421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4d7f7b90-55d6-437d-928e-f073d0cefae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:16.983721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 666/1318] 10.222.0.64 () {58 vars in 1060 bytes} [Wed Sep 21 16:14:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:16.992841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73aa250-8c7a-4480-a32f-c4279c662e8c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:16.994415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f73aa250-8c7a-4480-a32f-c4279c662e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:17.004160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:17.106577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73aa250-8c7a-4480-a32f-c4279c662e8c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:17.180409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeaeba5-4ff6-4ae4-ab66-dd72fd060f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:17.181015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaeba5-4ff6-4ae4-ab66-dd72fd060f7a service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:17.181294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaeba5-4ff6-4ae4-ab66-dd72fd060f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:17.181564 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeaeba5-4ff6-4ae4-ab66-dd72fd060f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:17.183199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2eeaeba5-4ff6-4ae4-ab66-dd72fd060f7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:17.247199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:17.294698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeaeba5-4ff6-4ae4-ab66-dd72fd060f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:17.296127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeaeba5-4ff6-4ae4-ab66-dd72fd060f7a service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:17.314937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 653/1319] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:16 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:14:20.414923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643db21b-a974-4561-993f-522bb85cfc3b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:20.415492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643db21b-a974-4561-993f-522bb85cfc3b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:20.415492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643db21b-a974-4561-993f-522bb85cfc3b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:20.416813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 667/1320] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:20.422636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a26bd1-373f-4297-b83f-49ce837b2788 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:20.423107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a26bd1-373f-4297-b83f-49ce837b2788 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:20.423249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a26bd1-373f-4297-b83f-49ce837b2788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:20.492409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a5a26bd1-373f-4297-b83f-49ce837b2788 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:20.493630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a5a26bd1-373f-4297-b83f-49ce837b2788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:20.589999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 654/1321] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:20.874667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643db21b-a974-4561-993f-522bb85cfc3b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:20.876301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-643db21b-a974-4561-993f-522bb85cfc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:20.928081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:20.984368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643db21b-a974-4561-993f-522bb85cfc3b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:21.060770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5436a1f-dc31-4fba-bc41-f53cab21873c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:21.061381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5436a1f-dc31-4fba-bc41-f53cab21873c service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:21.061655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5436a1f-dc31-4fba-bc41-f53cab21873c service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:21.061923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5436a1f-dc31-4fba-bc41-f53cab21873c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:21.063641 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d5436a1f-dc31-4fba-bc41-f53cab21873c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:21.159825 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:21.173553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5436a1f-dc31-4fba-bc41-f53cab21873c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:21.175009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5436a1f-dc31-4fba-bc41-f53cab21873c service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:21.192785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 668/1322] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:14:24.133689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd78e04-2ebf-435a-a613-fa8bb7ed4cf2 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:24.134274 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd78e04-2ebf-435a-a613-fa8bb7ed4cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:24.134274 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd78e04-2ebf-435a-a613-fa8bb7ed4cf2 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:24.135645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 655/1323] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:24.141237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77121c24-96d3-46c6-8061-30c8bdfa36b7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:24.141688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77121c24-96d3-46c6-8061-30c8bdfa36b7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:24.141784 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77121c24-96d3-46c6-8061-30c8bdfa36b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:24.211620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-77121c24-96d3-46c6-8061-30c8bdfa36b7 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:24.212843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-77121c24-96d3-46c6-8061-30c8bdfa36b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:24.313974 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 669/1324] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:24.601090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd78e04-2ebf-435a-a613-fa8bb7ed4cf2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:24.602812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2cd78e04-2ebf-435a-a613-fa8bb7ed4cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:24.625810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:24.716183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd78e04-2ebf-435a-a613-fa8bb7ed4cf2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:24.797655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceaab1c-8247-434c-a49c-a18175f267ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:24.798235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaab1c-8247-434c-a49c-a18175f267ce service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:24.798515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaab1c-8247-434c-a49c-a18175f267ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:24.798787 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceaab1c-8247-434c-a49c-a18175f267ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:24.800513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5ceaab1c-8247-434c-a49c-a18175f267ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:24.879315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:24.929117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaab1c-8247-434c-a49c-a18175f267ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:24.930749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceaab1c-8247-434c-a49c-a18175f267ce service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:24.951859 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 656/1325] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:14:27.240032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db392a57-7ecb-4847-ac54-8c9fd213056d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:27.240607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db392a57-7ecb-4847-ac54-8c9fd213056d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:27.240607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db392a57-7ecb-4847-ac54-8c9fd213056d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:27.242105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 670/1326] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:27.248254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f009fb2-582e-4af7-9434-761848c1a5df None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:27.248783 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f009fb2-582e-4af7-9434-761848c1a5df None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:27.248875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f009fb2-582e-4af7-9434-761848c1a5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:27.323924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1f009fb2-582e-4af7-9434-761848c1a5df None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:27.325162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1f009fb2-582e-4af7-9434-761848c1a5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:27.448929 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 657/1327] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:14:27.741345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db392a57-7ecb-4847-ac54-8c9fd213056d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:27.743642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-db392a57-7ecb-4847-ac54-8c9fd213056d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:27.811072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:27.858765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db392a57-7ecb-4847-ac54-8c9fd213056d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:27.947207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5c06f-c8e8-407d-910e-a6c44110d92b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:27.947843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5c06f-c8e8-407d-910e-a6c44110d92b service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:27.948122 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5c06f-c8e8-407d-910e-a6c44110d92b service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:27.948393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5c06f-c8e8-407d-910e-a6c44110d92b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:27.950102 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-78c5c06f-c8e8-407d-910e-a6c44110d92b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:28.082728 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:28.085361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5c06f-c8e8-407d-910e-a6c44110d92b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:28.087658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c5c06f-c8e8-407d-910e-a6c44110d92b service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:28.112171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 671/1328] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:14:30.407121 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980e217-960b-43c8-a573-0c8ec3a42e31 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:30.407664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980e217-960b-43c8-a573-0c8ec3a42e31 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:30.407664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980e217-960b-43c8-a573-0c8ec3a42e31 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:30.409020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 658/1329] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:30.414390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b767a3-29db-419f-a472-a44699c5076e None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:30.414805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b767a3-29db-419f-a472-a44699c5076e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:30.414941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b767a3-29db-419f-a472-a44699c5076e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:30.493952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-64b767a3-29db-419f-a472-a44699c5076e None None] MFA Rules not processed for user `1888f4b82c7940fdb9abfe0a57b404c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:30.495209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-64b767a3-29db-419f-a472-a44699c5076e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:30.609856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 672/1330] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:30.893877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980e217-960b-43c8-a573-0c8ec3a42e31 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:30.895560 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1980e217-960b-43c8-a573-0c8ec3a42e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:30.932108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:31.067331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980e217-960b-43c8-a573-0c8ec3a42e31 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:31.162474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3fef87-5c9a-4aab-a6c6-8767ccbf2b63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:31.163086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fef87-5c9a-4aab-a6c6-8767ccbf2b63 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:31.163366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fef87-5c9a-4aab-a6c6-8767ccbf2b63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:31.163638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3fef87-5c9a-4aab-a6c6-8767ccbf2b63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:31.165292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6d3fef87-5c9a-4aab-a6c6-8767ccbf2b63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:31.250006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:31.284630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fef87-5c9a-4aab-a6c6-8767ccbf2b63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:31.286087 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3fef87-5c9a-4aab-a6c6-8767ccbf2b63 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:31.305268 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 659/1331] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:30 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:14:33.782527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f97a4-b439-4127-87b6-3202bb876cc6 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:33.783034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f97a4-b439-4127-87b6-3202bb876cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:33.783034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f97a4-b439-4127-87b6-3202bb876cc6 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:33.784293 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 673/1332] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:33.789745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07e2e9-ff87-4b39-88d6-2b9d3d833e32 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:33.790171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07e2e9-ff87-4b39-88d6-2b9d3d833e32 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:33.790339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e07e2e9-ff87-4b39-88d6-2b9d3d833e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:33.857207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1e07e2e9-ff87-4b39-88d6-2b9d3d833e32 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:33.859056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1e07e2e9-ff87-4b39-88d6-2b9d3d833e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:33.956756 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 660/1333] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:34.241981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f97a4-b439-4127-87b6-3202bb876cc6 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:34.243583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c44f97a4-b439-4127-87b6-3202bb876cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:34.313393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:34.352081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f97a4-b439-4127-87b6-3202bb876cc6 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:34.427423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73173b-589b-463e-a851-96740bc34524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:34.428034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73173b-589b-463e-a851-96740bc34524 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:34.428313 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73173b-589b-463e-a851-96740bc34524 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:34.428581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73173b-589b-463e-a851-96740bc34524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:34.430253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-be73173b-589b-463e-a851-96740bc34524 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:34.542168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73173b-589b-463e-a851-96740bc34524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:34.543535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73173b-589b-463e-a851-96740bc34524 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:34.555674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:34.565976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 674/1334] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:14:36.784103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f53a3-95e3-4ccd-89b0-2c888638488a None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:36.784634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f53a3-95e3-4ccd-89b0-2c888638488a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:36.784634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f53a3-95e3-4ccd-89b0-2c888638488a None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:36.785899 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 661/1335] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:36.791191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b16309-ea2d-4275-875d-ebdf6dab05df None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:36.791605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b16309-ea2d-4275-875d-ebdf6dab05df None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:36.791740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b16309-ea2d-4275-875d-ebdf6dab05df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:36.860050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c7b16309-ea2d-4275-875d-ebdf6dab05df None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:36.861284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c7b16309-ea2d-4275-875d-ebdf6dab05df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:36.964317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 675/1336] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:37.256023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f53a3-95e3-4ccd-89b0-2c888638488a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:37.257558 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8c0f53a3-95e3-4ccd-89b0-2c888638488a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:37.298368 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:37.367258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f53a3-95e3-4ccd-89b0-2c888638488a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:37.442503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f13b44-d656-4a0b-b1b8-3627e194350d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:37.443109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f13b44-d656-4a0b-b1b8-3627e194350d service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:37.443389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f13b44-d656-4a0b-b1b8-3627e194350d service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:37.443663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f13b44-d656-4a0b-b1b8-3627e194350d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:37.445270 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-05f13b44-d656-4a0b-b1b8-3627e194350d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:37.533243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:37.559883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f13b44-d656-4a0b-b1b8-3627e194350d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:37.561370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f13b44-d656-4a0b-b1b8-3627e194350d service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:37.579605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 662/1337] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:14:40.130439 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68afcbb-d180-4bd1-b1c1-597180e03e47 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:40.130984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68afcbb-d180-4bd1-b1c1-597180e03e47 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:40.130984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68afcbb-d180-4bd1-b1c1-597180e03e47 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:40.132309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 676/1338] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:40.137634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd26376-f2b6-4327-ba73-d77e9d07feb2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:40.138066 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd26376-f2b6-4327-ba73-d77e9d07feb2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:40.138250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd26376-f2b6-4327-ba73-d77e9d07feb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:40.205265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ccd26376-f2b6-4327-ba73-d77e9d07feb2 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:40.206512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ccd26376-f2b6-4327-ba73-d77e9d07feb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:40.302652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 663/1339] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:40.590684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68afcbb-d180-4bd1-b1c1-597180e03e47 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:40.592278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a68afcbb-d180-4bd1-b1c1-597180e03e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:40.679533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:40.708615 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68afcbb-d180-4bd1-b1c1-597180e03e47 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:40.796453 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a65a17-e051-4bbb-a0f1-d911fd6eee75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:40.796972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65a17-e051-4bbb-a0f1-d911fd6eee75 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:40.797251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65a17-e051-4bbb-a0f1-d911fd6eee75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:40.797524 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65a17-e051-4bbb-a0f1-d911fd6eee75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:40.799275 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-32a65a17-e051-4bbb-a0f1-d911fd6eee75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:40.916453 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a65a17-e051-4bbb-a0f1-d911fd6eee75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:40.917889 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a65a17-e051-4bbb-a0f1-d911fd6eee75 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:40.937353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 677/1340] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:14:45.220309 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016baa36-079d-45a2-b70e-7e6dc3d967ee None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:45.221256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016baa36-079d-45a2-b70e-7e6dc3d967ee None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:45.221689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016baa36-079d-45a2-b70e-7e6dc3d967ee None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:45.223294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 664/1341] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:45.229368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21b0ab-37ce-4c67-bab0-d36f3386b662 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:45.229915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21b0ab-37ce-4c67-bab0-d36f3386b662 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:45.229915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb21b0ab-37ce-4c67-bab0-d36f3386b662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:45.236745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:45.305955 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-eb21b0ab-37ce-4c67-bab0-d36f3386b662 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:45.307118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-eb21b0ab-37ce-4c67-bab0-d36f3386b662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:45.417132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 678/1342] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:45.712367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016baa36-079d-45a2-b70e-7e6dc3d967ee None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:45.713901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-016baa36-079d-45a2-b70e-7e6dc3d967ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:45.768914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:45.835131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016baa36-079d-45a2-b70e-7e6dc3d967ee None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:45.942140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ca17d-e4f2-4184-b1ff-66c5dccb1756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:45.942798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ca17d-e4f2-4184-b1ff-66c5dccb1756 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:45.943049 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ca17d-e4f2-4184-b1ff-66c5dccb1756 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:45.945330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ca17d-e4f2-4184-b1ff-66c5dccb1756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:45.947091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ff5ca17d-e4f2-4184-b1ff-66c5dccb1756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:46.060524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:46.082129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ca17d-e4f2-4184-b1ff-66c5dccb1756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:46.083597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ca17d-e4f2-4184-b1ff-66c5dccb1756 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:46.104114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 665/1343] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:14:49.247504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5821cc8-c5df-4a33-89f5-e9e3c8200b27 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:49.248105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5821cc8-c5df-4a33-89f5-e9e3c8200b27 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:49.248105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5821cc8-c5df-4a33-89f5-e9e3c8200b27 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:49.249563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 679/1344] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:49.255961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7afe2-a25e-472a-8017-9e6a55392a59 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:49.256498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7afe2-a25e-472a-8017-9e6a55392a59 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:49.256498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7afe2-a25e-472a-8017-9e6a55392a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:49.337828 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-44e7afe2-a25e-472a-8017-9e6a55392a59 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:49.339148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-44e7afe2-a25e-472a-8017-9e6a55392a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:49.443697 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 666/1345] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:49.731663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5821cc8-c5df-4a33-89f5-e9e3c8200b27 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:49.733245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d5821cc8-c5df-4a33-89f5-e9e3c8200b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:49.828118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:49.847147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5821cc8-c5df-4a33-89f5-e9e3c8200b27 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:49.945317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0931c6-5547-46a6-a873-0c7a3e9419b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:49.945981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0931c6-5547-46a6-a873-0c7a3e9419b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:49.946354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0931c6-5547-46a6-a873-0c7a3e9419b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:49.946737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0931c6-5547-46a6-a873-0c7a3e9419b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:49.948504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ca0931c6-5547-46a6-a873-0c7a3e9419b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:50.103934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0931c6-5547-46a6-a873-0c7a3e9419b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:50.105449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0931c6-5547-46a6-a873-0c7a3e9419b1 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:50.126363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 680/1346] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:14:54.612144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db521be-2e2a-414c-bfb8-02ecf7bfb132 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:54.612144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db521be-2e2a-414c-bfb8-02ecf7bfb132 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:54.612144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db521be-2e2a-414c-bfb8-02ecf7bfb132 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:54.612144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 667/1347] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:14:54.617683 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce7a52-398c-4f99-ad39-4bacff31b2b0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:54.618276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce7a52-398c-4f99-ad39-4bacff31b2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:54.618392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ce7a52-398c-4f99-ad39-4bacff31b2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:54.640719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:54.703189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-74ce7a52-398c-4f99-ad39-4bacff31b2b0 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:14:54.704434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-74ce7a52-398c-4f99-ad39-4bacff31b2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:54.835692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 681/1348] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:14:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:14:55.131686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db521be-2e2a-414c-bfb8-02ecf7bfb132 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:14:55.134260 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7db521be-2e2a-414c-bfb8-02ecf7bfb132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:55.201673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:55.250821 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db521be-2e2a-414c-bfb8-02ecf7bfb132 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:14:55.329711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3590a7-0670-4d85-97ae-e99efe1c8400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:14:55.330353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3590a7-0670-4d85-97ae-e99efe1c8400 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:14:55.330622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3590a7-0670-4d85-97ae-e99efe1c8400 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:14:55.330887 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3590a7-0670-4d85-97ae-e99efe1c8400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:14:55.332501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fc3590a7-0670-4d85-97ae-e99efe1c8400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:14:55.440245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:14:55.444534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3590a7-0670-4d85-97ae-e99efe1c8400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:14:55.445965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3590a7-0670-4d85-97ae-e99efe1c8400 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:14:55.465031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 668/1349] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:14:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:15:00.674676 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0513bf-fcfd-48aa-b9aa-0082fd9e3fc7 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:00.675469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0513bf-fcfd-48aa-b9aa-0082fd9e3fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:00.675925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0513bf-fcfd-48aa-b9aa-0082fd9e3fc7 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:00.677797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 682/1350] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:15:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:15:00.686389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485161b-e5c0-4264-b53e-6af6a4939f75 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:00.686389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485161b-e5c0-4264-b53e-6af6a4939f75 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:00.686389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1485161b-e5c0-4264-b53e-6af6a4939f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:00.877252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1485161b-e5c0-4264-b53e-6af6a4939f75 None None] MFA Rules not processed for user `55642294e8ff4f779510f4646e0e30a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:00.879217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1485161b-e5c0-4264-b53e-6af6a4939f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:01.129385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 669/1351] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:15:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:04.834214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=59 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:06.893778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c8f0f-2a15-4a37-92ca-566d60a2c57d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:06.896147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c8f0f-2a15-4a37-92ca-566d60a2c57d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:06.897121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c8f0f-2a15-4a37-92ca-566d60a2c57d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:06.899948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 683/1352] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:15:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:06.909725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5a211-1e9c-48a7-a3f3-7441e2a8a3d5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:06.910346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5a211-1e9c-48a7-a3f3-7441e2a8a3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:06.910346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5a211-1e9c-48a7-a3f3-7441e2a8a3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:06.984288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:07.019693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-21f5a211-1e9c-48a7-a3f3-7441e2a8a3d5 None None] MFA Rules not processed for user `55642294e8ff4f779510f4646e0e30a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:07.022141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-21f5a211-1e9c-48a7-a3f3-7441e2a8a3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:07.160145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 670/1353] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:15:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:09.149640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ffa6e-6a1b-4ffa-b117-a2641bf2a15d None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:09.150935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ffa6e-6a1b-4ffa-b117-a2641bf2a15d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:09.150935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ffa6e-6a1b-4ffa-b117-a2641bf2a15d None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:09.152965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 684/1354] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:09.160666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41eb3a7-f1aa-4711-8ff2-a206d17c4337 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:09.161518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41eb3a7-f1aa-4711-8ff2-a206d17c4337 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:09.161518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41eb3a7-f1aa-4711-8ff2-a206d17c4337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:09.268826 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b41eb3a7-f1aa-4711-8ff2-a206d17c4337 None None] MFA Rules not processed for user `55642294e8ff4f779510f4646e0e30a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:09.270972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b41eb3a7-f1aa-4711-8ff2-a206d17c4337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:09.335462 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:09.421463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 671/1355] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:15:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:13.012434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9258a0-97a4-4906-a766-2d81c284c26f None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:13.013113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9258a0-97a4-4906-a766-2d81c284c26f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:13.013626 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9258a0-97a4-4906-a766-2d81c284c26f None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:13.016522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 685/1356] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:13.024796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5ab5a-0e6f-4774-b11e-9ed463f378bc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:13.026353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5ab5a-0e6f-4774-b11e-9ed463f378bc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:13.027067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5ab5a-0e6f-4774-b11e-9ed463f378bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:13.153419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-71a5ab5a-0e6f-4774-b11e-9ed463f378bc None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:13.154849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-71a5ab5a-0e6f-4774-b11e-9ed463f378bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:13.272666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 672/1357] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:13.592480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56494cf0-b727-4656-ac0c-97551b36537c None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:13.593224 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56494cf0-b727-4656-ac0c-97551b36537c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:13.593604 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56494cf0-b727-4656-ac0c-97551b36537c None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:13.595133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 686/1358] 10.222.0.64 () {54 vars in 965 bytes} [Wed Sep 21 16:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:15:13.610650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4fbd6-f814-4dd1-b703-33371cbbcc99 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:13.611292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4fbd6-f814-4dd1-b703-33371cbbcc99 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:13.611803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef4fbd6-f814-4dd1-b703-33371cbbcc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:13.646183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:13.774556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5ef4fbd6-f814-4dd1-b703-33371cbbcc99 None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:13.776948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5ef4fbd6-f814-4dd1-b703-33371cbbcc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:14.001485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 673/1359] 10.222.0.64 () {58 vars in 1062 bytes} [Wed Sep 21 16:15:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:15:14.014640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56494cf0-b727-4656-ac0c-97551b36537c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:14.016583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-56494cf0-b727-4656-ac0c-97551b36537c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:14.219034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56494cf0-b727-4656-ac0c-97551b36537c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:14.327493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:14.390135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6baef3-7cfb-4fec-9eff-43f30001edca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:14.390956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6baef3-7cfb-4fec-9eff-43f30001edca service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:14.391343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6baef3-7cfb-4fec-9eff-43f30001edca service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:14.391708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6baef3-7cfb-4fec-9eff-43f30001edca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:14.393635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1e6baef3-7cfb-4fec-9eff-43f30001edca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:14.515856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6baef3-7cfb-4fec-9eff-43f30001edca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:14.519243 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6baef3-7cfb-4fec-9eff-43f30001edca service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:14.554728 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 687/1360] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:15:17.252428 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eabc0-8d35-492d-a616-a6df978a29f7 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:17.252983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eabc0-8d35-492d-a616-a6df978a29f7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:17.253672 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00eabc0-8d35-492d-a616-a6df978a29f7 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:17.256331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 674/1361] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:17.263590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491cd43-b832-431d-a3fb-8d6864ea1cf9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:17.264500 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491cd43-b832-431d-a3fb-8d6864ea1cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:17.264500 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491cd43-b832-431d-a3fb-8d6864ea1cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:17.422601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5491cd43-b832-431d-a3fb-8d6864ea1cf9 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:17.424498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5491cd43-b832-431d-a3fb-8d6864ea1cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:17.490014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:17.582175 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 688/1362] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:15:17.978647 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00eabc0-8d35-492d-a616-a6df978a29f7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:17.980957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e00eabc0-8d35-492d-a616-a6df978a29f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:18.064714 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:18.164744 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00eabc0-8d35-492d-a616-a6df978a29f7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:18.309689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22083929-0e2b-46cf-a95f-42c9000d1818 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:18.310416 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22083929-0e2b-46cf-a95f-42c9000d1818 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:18.310618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22083929-0e2b-46cf-a95f-42c9000d1818 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:18.310892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22083929-0e2b-46cf-a95f-42c9000d1818 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:18.312766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-22083929-0e2b-46cf-a95f-42c9000d1818 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:18.343226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:18.434435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22083929-0e2b-46cf-a95f-42c9000d1818 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:18.436128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22083929-0e2b-46cf-a95f-42c9000d1818 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:18.458367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 675/1363] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:15:21.088790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998545cf-8aec-4d0f-bae8-e47747baebda None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:21.090345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998545cf-8aec-4d0f-bae8-e47747baebda None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:21.090345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998545cf-8aec-4d0f-bae8-e47747baebda None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:21.092406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 689/1364] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:21.098757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5174a09-f519-45f9-9023-ac227e13dc73 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:21.099497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5174a09-f519-45f9-9023-ac227e13dc73 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:21.099694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5174a09-f519-45f9-9023-ac227e13dc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:21.191133 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b5174a09-f519-45f9-9023-ac227e13dc73 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:21.193283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b5174a09-f519-45f9-9023-ac227e13dc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:21.376114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:21.390249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 676/1365] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:15:21.898425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3bfac-b5a6-45d7-95d2-30db7d4f6315 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:21.898861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3bfac-b5a6-45d7-95d2-30db7d4f6315 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:21.899379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce3bfac-b5a6-45d7-95d2-30db7d4f6315 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:21.902090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 690/1366] 10.222.0.64 () {54 vars in 965 bytes} [Wed Sep 21 16:15:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:21.908746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e01ad1-b82a-487a-97ed-f46b3d0db22f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:21.909219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e01ad1-b82a-487a-97ed-f46b3d0db22f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:21.909570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e01ad1-b82a-487a-97ed-f46b3d0db22f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:22.029328 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-26e01ad1-b82a-487a-97ed-f46b3d0db22f None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:22.032204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-26e01ad1-b82a-487a-97ed-f46b3d0db22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:22.224461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 677/1367] 10.222.0.64 () {58 vars in 1062 bytes} [Wed Sep 21 16:15:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:15:22.240565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3bfac-b5a6-45d7-95d2-30db7d4f6315 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:22.243667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-dce3bfac-b5a6-45d7-95d2-30db7d4f6315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:22.500819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce3bfac-b5a6-45d7-95d2-30db7d4f6315 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:22.529974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1523df-e35d-4fec-8f2e-a12e16a43190 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:22.530288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1523df-e35d-4fec-8f2e-a12e16a43190 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:22.530570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1523df-e35d-4fec-8f2e-a12e16a43190 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:22.531989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 678/1368] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:15:22.537536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be93c1-040e-474e-a1a7-b01b28532120 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:22.537811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be93c1-040e-474e-a1a7-b01b28532120 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:22.538262 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be93c1-040e-474e-a1a7-b01b28532120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:22.547051 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:22.652941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:22.668117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-25be93c1-040e-474e-a1a7-b01b28532120 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:22.670835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-25be93c1-040e-474e-a1a7-b01b28532120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:22.716034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d0d6a-bbcf-431e-b312-fa0571d18b65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:22.717030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d0d6a-bbcf-431e-b312-fa0571d18b65 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:22.717457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d0d6a-bbcf-431e-b312-fa0571d18b65 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:22.717741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d0d6a-bbcf-431e-b312-fa0571d18b65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:22.719767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ae3d0d6a-bbcf-431e-b312-fa0571d18b65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:22.880372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 679/1369] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:15:22.903543 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d0d6a-bbcf-431e-b312-fa0571d18b65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:22.906414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d0d6a-bbcf-431e-b312-fa0571d18b65 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:22.938883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 691/1370] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:15:23.197056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ba07d-7895-4170-9a6c-280a9ffceeb2 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:23.197551 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ba07d-7895-4170-9a6c-280a9ffceeb2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:23.198285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ba07d-7895-4170-9a6c-280a9ffceeb2 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:23.201554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 680/1371] 10.222.0.64 () {54 vars in 963 bytes} [Wed Sep 21 16:15:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:23.207786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f59539-0e56-4c87-8304-ee363c33901a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:23.208447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f59539-0e56-4c87-8304-ee363c33901a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:23.208934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f59539-0e56-4c87-8304-ee363c33901a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:23.246337 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:23.311434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e3f59539-0e56-4c87-8304-ee363c33901a None None] MFA Rules not processed for user `e9fb56e56c7f49ce959d0bc6c515044c`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:23.312580 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e3f59539-0e56-4c87-8304-ee363c33901a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:23.422396 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 692/1372] 10.222.0.64 () {58 vars in 1060 bytes} [Wed Sep 21 16:15:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:23.426729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ba07d-7895-4170-9a6c-280a9ffceeb2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:23.428491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-191ba07d-7895-4170-9a6c-280a9ffceeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:23.586607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ba07d-7895-4170-9a6c-280a9ffceeb2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:23.626403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:23.686901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b794909-99b7-4ccc-b754-2d3001d0c56f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:23.687528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b794909-99b7-4ccc-b754-2d3001d0c56f service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:23.687805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b794909-99b7-4ccc-b754-2d3001d0c56f service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:23.688085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b794909-99b7-4ccc-b754-2d3001d0c56f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:23.689798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0b794909-99b7-4ccc-b754-2d3001d0c56f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:23.839810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b794909-99b7-4ccc-b754-2d3001d0c56f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:23.841290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b794909-99b7-4ccc-b754-2d3001d0c56f service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:23.869276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 681/1373] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:15:25.415178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9cbde-2cac-4a03-ae96-586f6d351add None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:25.415920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9cbde-2cac-4a03-ae96-586f6d351add None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:25.415920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9cbde-2cac-4a03-ae96-586f6d351add None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:25.417581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 693/1374] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:25.428363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b38af3-23f1-4342-9230-87f1dfed3992 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:25.429523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b38af3-23f1-4342-9230-87f1dfed3992 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:25.429973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b38af3-23f1-4342-9230-87f1dfed3992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:25.517570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-40b38af3-23f1-4342-9230-87f1dfed3992 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:25.519972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-40b38af3-23f1-4342-9230-87f1dfed3992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:25.532313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:25.647593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 682/1375] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:25.946647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9cbde-2cac-4a03-ae96-586f6d351add None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:25.948345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-33b9cbde-2cac-4a03-ae96-586f6d351add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:26.099446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:26.141386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9cbde-2cac-4a03-ae96-586f6d351add None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:26.285528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d5cb6-1a6a-41de-af66-fdc1a1568618 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:26.286202 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d5cb6-1a6a-41de-af66-fdc1a1568618 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:26.286488 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d5cb6-1a6a-41de-af66-fdc1a1568618 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:26.286757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d5cb6-1a6a-41de-af66-fdc1a1568618 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:26.288520 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8f2d5cb6-1a6a-41de-af66-fdc1a1568618 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:26.420801 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:26.437394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d5cb6-1a6a-41de-af66-fdc1a1568618 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:26.439142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d5cb6-1a6a-41de-af66-fdc1a1568618 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:26.462292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 694/1376] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:15:28.853540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3704566-dc63-4167-8189-b93f9de44c53 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:28.855480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3704566-dc63-4167-8189-b93f9de44c53 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:28.855480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3704566-dc63-4167-8189-b93f9de44c53 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:28.860352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 683/1377] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:15:28.861812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad46dc8-cbb2-4a3d-946a-a33eebc95782 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:28.862470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad46dc8-cbb2-4a3d-946a-a33eebc95782 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:28.862470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad46dc8-cbb2-4a3d-946a-a33eebc95782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:28.942873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2ad46dc8-cbb2-4a3d-946a-a33eebc95782 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:28.944208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2ad46dc8-cbb2-4a3d-946a-a33eebc95782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:29.077690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 695/1378] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:15:29.379504 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3704566-dc63-4167-8189-b93f9de44c53 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:29.382928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a3704566-dc63-4167-8189-b93f9de44c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:29.653325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:29.673369 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3704566-dc63-4167-8189-b93f9de44c53 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:29.905398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71257a7-0f52-4aa4-8667-77e81e7fe204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:29.906446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71257a7-0f52-4aa4-8667-77e81e7fe204 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:29.906874 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71257a7-0f52-4aa4-8667-77e81e7fe204 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:29.907292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71257a7-0f52-4aa4-8667-77e81e7fe204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:29.910337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b71257a7-0f52-4aa4-8667-77e81e7fe204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:30.079117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71257a7-0f52-4aa4-8667-77e81e7fe204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:30.080858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71257a7-0f52-4aa4-8667-77e81e7fe204 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:30.095302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:30.108628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 684/1379] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 730 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:15:32.598503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f7e02-735c-438a-aa79-261ec85cba0f None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:32.599098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f7e02-735c-438a-aa79-261ec85cba0f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:32.599436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f7e02-735c-438a-aa79-261ec85cba0f None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:32.601462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 696/1380] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:32.608832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204228a-6375-4b90-87fb-2496cb035d7f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:32.609508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204228a-6375-4b90-87fb-2496cb035d7f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:32.610210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204228a-6375-4b90-87fb-2496cb035d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:32.707905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0204228a-6375-4b90-87fb-2496cb035d7f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:32.709460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0204228a-6375-4b90-87fb-2496cb035d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:32.884179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 685/1381] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:33.292644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f7e02-735c-438a-aa79-261ec85cba0f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:33.294913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ec7f7e02-735c-438a-aa79-261ec85cba0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:33.386251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:33.504297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f7e02-735c-438a-aa79-261ec85cba0f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:33.668638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2b91b-15ce-407f-a53a-9de639344497 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:33.669699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2b91b-15ce-407f-a53a-9de639344497 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:33.669699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2b91b-15ce-407f-a53a-9de639344497 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:33.669927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2b91b-15ce-407f-a53a-9de639344497 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:33.671476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6ef2b91b-15ce-407f-a53a-9de639344497 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:33.703508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:33.797668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2b91b-15ce-407f-a53a-9de639344497 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:33.799382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2b91b-15ce-407f-a53a-9de639344497 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:33.833217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 697/1382] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:15:36.233332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab5021-bcfa-4b6d-a6e4-a1174cc7c100 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:36.233948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab5021-bcfa-4b6d-a6e4-a1174cc7c100 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:36.233948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab5021-bcfa-4b6d-a6e4-a1174cc7c100 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:36.235252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 686/1383] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:36.240998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80d8b3-9642-4d7c-9418-4f5bcdbe5a0c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:36.241545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80d8b3-9642-4d7c-9418-4f5bcdbe5a0c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:36.241545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80d8b3-9642-4d7c-9418-4f5bcdbe5a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:36.318670 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3c80d8b3-9642-4d7c-9418-4f5bcdbe5a0c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:36.319919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3c80d8b3-9642-4d7c-9418-4f5bcdbe5a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:36.431843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:36.438845 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 698/1384] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:36.739828 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab5021-bcfa-4b6d-a6e4-a1174cc7c100 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:36.741250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-76ab5021-bcfa-4b6d-a6e4-a1174cc7c100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:36.840655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:36.947869 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab5021-bcfa-4b6d-a6e4-a1174cc7c100 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:37.104627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8218a2-f8bf-417b-9427-ce39a775fc0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:37.105649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8218a2-f8bf-417b-9427-ce39a775fc0d service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:37.106163 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8218a2-f8bf-417b-9427-ce39a775fc0d service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:37.106524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8218a2-f8bf-417b-9427-ce39a775fc0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:37.108880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0e8218a2-f8bf-417b-9427-ce39a775fc0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:37.164866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:37.242083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8218a2-f8bf-417b-9427-ce39a775fc0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:37.243725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8218a2-f8bf-417b-9427-ce39a775fc0d service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:37.272791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 687/1385] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:15:39.657720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67af1e-93c0-4175-be02-15d66bc4a512 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:39.658349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67af1e-93c0-4175-be02-15d66bc4a512 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:39.658489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d67af1e-93c0-4175-be02-15d66bc4a512 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:39.660811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 699/1386] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:39.667711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579612d7-1ee4-435d-bbbc-d2671fdbffaf None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:39.668654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579612d7-1ee4-435d-bbbc-d2671fdbffaf None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:39.668654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579612d7-1ee4-435d-bbbc-d2671fdbffaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:39.764689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-579612d7-1ee4-435d-bbbc-d2671fdbffaf None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:39.765864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-579612d7-1ee4-435d-bbbc-d2671fdbffaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:39.874999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 688/1387] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:15:40.169503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67af1e-93c0-4175-be02-15d66bc4a512 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:40.171982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9d67af1e-93c0-4175-be02-15d66bc4a512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:40.391965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d67af1e-93c0-4175-be02-15d66bc4a512 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:40.496011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:40.575899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f933228-79e9-41f3-b6f5-ef6d20b2d57e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:40.576950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f933228-79e9-41f3-b6f5-ef6d20b2d57e service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:40.577440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f933228-79e9-41f3-b6f5-ef6d20b2d57e service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:40.577736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f933228-79e9-41f3-b6f5-ef6d20b2d57e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:40.580790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6f933228-79e9-41f3-b6f5-ef6d20b2d57e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:40.697823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f933228-79e9-41f3-b6f5-ef6d20b2d57e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:40.700589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f933228-79e9-41f3-b6f5-ef6d20b2d57e service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:40.736754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 700/1388] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:15:43.059113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09828b21-4075-4cfe-85be-819b012a7692 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:43.059919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09828b21-4075-4cfe-85be-819b012a7692 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:43.059919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09828b21-4075-4cfe-85be-819b012a7692 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:43.061546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 689/1389] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:43.068077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addd28e-8266-4fb4-9479-2d662edfb146 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:43.068901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addd28e-8266-4fb4-9479-2d662edfb146 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:43.068901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addd28e-8266-4fb4-9479-2d662edfb146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:43.158232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7addd28e-8266-4fb4-9479-2d662edfb146 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:43.159494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7addd28e-8266-4fb4-9479-2d662edfb146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:43.199075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:43.264994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94c6f5-3f0d-4775-99a7-a59ba12c82f9 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:43.265280 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94c6f5-3f0d-4775-99a7-a59ba12c82f9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:43.265555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94c6f5-3f0d-4775-99a7-a59ba12c82f9 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:43.267072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 690/1390] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:15:43.272403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923db24-e0d5-482a-b164-a50e245b1ec5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:43.272638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 701/1391] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:43.272784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923db24-e0d5-482a-b164-a50e245b1ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:43.273001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923db24-e0d5-482a-b164-a50e245b1ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:43.289340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:43.351332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2923db24-e0d5-482a-b164-a50e245b1ec5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:43.352478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2923db24-e0d5-482a-b164-a50e245b1ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:43.462276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 691/1392] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:15:43.575025 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addd28e-8266-4fb4-9479-2d662edfb146 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:43.576251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7addd28e-8266-4fb4-9479-2d662edfb146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:43.700974 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addd28e-8266-4fb4-9479-2d662edfb146 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:43.756564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:43.776103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4c3d2-52dc-459d-bb36-14830681c900 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:43.776479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4c3d2-52dc-459d-bb36-14830681c900 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:43.776823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4c3d2-52dc-459d-bb36-14830681c900 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:43.778460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 692/1393] 10.222.0.64 () {54 vars in 962 bytes} [Wed Sep 21 16:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:15:43.785655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d3f83-c86d-47f5-b893-141d4da84035 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:43.786332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d3f83-c86d-47f5-b893-141d4da84035 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:43.786904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d3f83-c86d-47f5-b893-141d4da84035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:43.832309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73196a92-16d5-4b81-bec6-d675a811a151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:43.833022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73196a92-16d5-4b81-bec6-d675a811a151 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:43.833300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73196a92-16d5-4b81-bec6-d675a811a151 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:43.833662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73196a92-16d5-4b81-bec6-d675a811a151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:43.835376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-73196a92-16d5-4b81-bec6-d675a811a151 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:43.878664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-af3d3f83-c86d-47f5-b893-141d4da84035 None None] MFA Rules not processed for user `d2d76fb9cbdd40f9b2a748209cee5b75`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:43.879920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-af3d3f83-c86d-47f5-b893-141d4da84035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:43.924120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:43.950321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73196a92-16d5-4b81-bec6-d675a811a151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:43.952306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73196a92-16d5-4b81-bec6-d675a811a151 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:43.973735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 693/1394] 10.222.0.64 () {58 vars in 1059 bytes} [Wed Sep 21 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:15:43.978872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 702/1395] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:15:43.983699 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d3f83-c86d-47f5-b893-141d4da84035 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:43.993468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d3f83-c86d-47f5-b893-141d4da84035 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:44.090392 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53340c69-da55-4140-b6e4-df2505ee6df7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2d76fb9cbdd40f9b2a748209cee5b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:44.091241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53340c69-da55-4140-b6e4-df2505ee6df7 service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:44.091334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53340c69-da55-4140-b6e4-df2505ee6df7 service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:44.091550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53340c69-da55-4140-b6e4-df2505ee6df7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:44.093252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-53340c69-da55-4140-b6e4-df2505ee6df7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:44.218028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:44.222684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53340c69-da55-4140-b6e4-df2505ee6df7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:44.224142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53340c69-da55-4140-b6e4-df2505ee6df7 service glance] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:44.246069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 694/1396] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:15:45.025565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d4bd2-b4f7-48db-baf4-31963eb5ffd1 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:45.026205 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d4bd2-b4f7-48db-baf4-31963eb5ffd1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:45.026205 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d4bd2-b4f7-48db-baf4-31963eb5ffd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:45.097426 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-bd9d4bd2-b4f7-48db-baf4-31963eb5ffd1 None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:45.098699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-bd9d4bd2-b4f7-48db-baf4-31963eb5ffd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:45.101623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:45.192778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 703/1397] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:15:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:45.240292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773698b-3e3b-4680-8f08-9acc1f35b923 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:45.240835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773698b-3e3b-4680-8f08-9acc1f35b923 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:45.240835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773698b-3e3b-4680-8f08-9acc1f35b923 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:45.242337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 695/1398] 10.222.0.64 () {54 vars in 943 bytes} [Wed Sep 21 16:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:45.250665 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e9b03-708b-4191-b2f3-7e7c899b8b59 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:45.250954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e9b03-708b-4191-b2f3-7e7c899b8b59 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:45.251224 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e9b03-708b-4191-b2f3-7e7c899b8b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:45.352553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-532e9b03-708b-4191-b2f3-7e7c899b8b59 None None] MFA Rules not processed for user `d71a1b8081ce490db46db95f14024ba6`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:45.353719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-532e9b03-708b-4191-b2f3-7e7c899b8b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:45.439150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:45.473739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 704/1399] 10.222.0.64 () {58 vars in 1040 bytes} [Wed Sep 21 16:15:45 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:45.485988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773698b-3e3b-4680-8f08-9acc1f35b923 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:45.488931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6773698b-3e3b-4680-8f08-9acc1f35b923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:45.647668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773698b-3e3b-4680-8f08-9acc1f35b923 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:45.721905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:45.759952 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62b3e6-c157-47cb-b185-40447c59f400 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:45.760563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b3e6-c157-47cb-b185-40447c59f400 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:45.760840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b3e6-c157-47cb-b185-40447c59f400 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:45.761109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b3e6-c157-47cb-b185-40447c59f400 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:45.762795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5e62b3e6-c157-47cb-b185-40447c59f400 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:45.851315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62b3e6-c157-47cb-b185-40447c59f400 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:45.852967 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e62b3e6-c157-47cb-b185-40447c59f400 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:45.862171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 696/1400] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:15:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 16:15:46.288009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bf671-f500-40bb-b89b-c344c02240ce None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:46.288548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bf671-f500-40bb-b89b-c344c02240ce None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:46.288548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bf671-f500-40bb-b89b-c344c02240ce None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:46.289817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 705/1401] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:46.296372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af542b65-09ee-481d-b234-d4477c8dedd8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:46.296372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af542b65-09ee-481d-b234-d4477c8dedd8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:46.297165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af542b65-09ee-481d-b234-d4477c8dedd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:46.392666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-af542b65-09ee-481d-b234-d4477c8dedd8 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:46.394088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-af542b65-09ee-481d-b234-d4477c8dedd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:46.469910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:46.530097 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 697/1402] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:46.856364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bf671-f500-40bb-b89b-c344c02240ce None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:46.858094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b75bf671-f500-40bb-b89b-c344c02240ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:47.094750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bf671-f500-40bb-b89b-c344c02240ce None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:47.183316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:47.271735 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7e85f-b526-4d08-98ec-1133f59586c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:47.273075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7e85f-b526-4d08-98ec-1133f59586c2 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:47.273541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7e85f-b526-4d08-98ec-1133f59586c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:47.273982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7e85f-b526-4d08-98ec-1133f59586c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:47.276015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3ef7e85f-b526-4d08-98ec-1133f59586c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:47.424860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7e85f-b526-4d08-98ec-1133f59586c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:47.426553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7e85f-b526-4d08-98ec-1133f59586c2 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:47.449184 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 706/1403] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:15:49.917754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aca544-b6bf-4c2d-b57b-05b07a025755 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:49.918514 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aca544-b6bf-4c2d-b57b-05b07a025755 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:49.918740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aca544-b6bf-4c2d-b57b-05b07a025755 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:49.927696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 698/1404] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:49 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:15:49.928320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb2c53-c671-4eb6-969e-be69f9575cf0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:49.929126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb2c53-c671-4eb6-969e-be69f9575cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:49.929126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeb2c53-c671-4eb6-969e-be69f9575cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:50.032617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-caeb2c53-c671-4eb6-969e-be69f9575cf0 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:50.037990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-caeb2c53-c671-4eb6-969e-be69f9575cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:50.073319 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:50.206185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 707/1405] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:50.543280 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aca544-b6bf-4c2d-b57b-05b07a025755 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:50.544833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d1aca544-b6bf-4c2d-b57b-05b07a025755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:50.792160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aca544-b6bf-4c2d-b57b-05b07a025755 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:50.871966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:51.056860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173d974-3c3a-4eef-b133-9a60f793609a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:51.058495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173d974-3c3a-4eef-b133-9a60f793609a service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:51.059642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173d974-3c3a-4eef-b133-9a60f793609a service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:51.060184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173d974-3c3a-4eef-b133-9a60f793609a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:51.063269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9173d974-3c3a-4eef-b133-9a60f793609a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:51.227405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173d974-3c3a-4eef-b133-9a60f793609a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:51.229726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173d974-3c3a-4eef-b133-9a60f793609a service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:51.258095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 699/1406] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 712 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:15:53.697923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769f3b4-6a51-41b4-ad25-c3a1a7abcab3 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:53.698809 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769f3b4-6a51-41b4-ad25-c3a1a7abcab3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:53.698809 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769f3b4-6a51-41b4-ad25-c3a1a7abcab3 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:53.702856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 708/1407] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:15:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:15:53.705586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243db05-ee2a-4ef1-9353-9a2392f829be None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:53.706155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243db05-ee2a-4ef1-9353-9a2392f829be None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:53.706155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243db05-ee2a-4ef1-9353-9a2392f829be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:53.781358 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:53.793734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9243db05-ee2a-4ef1-9353-9a2392f829be None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:15:53.795020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9243db05-ee2a-4ef1-9353-9a2392f829be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:53.996537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 700/1408] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:15:54.302667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769f3b4-6a51-41b4-ad25-c3a1a7abcab3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:15:54.304325 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a769f3b4-6a51-41b4-ad25-c3a1a7abcab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:54.537663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:15:54.547104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769f3b4-6a51-41b4-ad25-c3a1a7abcab3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:15:54.759977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe129fe6-dd4c-461b-b1dc-e80ce65da8d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:15:54.760793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe129fe6-dd4c-461b-b1dc-e80ce65da8d3 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:15:54.760949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe129fe6-dd4c-461b-b1dc-e80ce65da8d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:15:54.761285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe129fe6-dd4c-461b-b1dc-e80ce65da8d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:15:54.762950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fe129fe6-dd4c-461b-b1dc-e80ce65da8d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:15:54.889283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe129fe6-dd4c-461b-b1dc-e80ce65da8d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:15:54.892198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe129fe6-dd4c-461b-b1dc-e80ce65da8d3 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:15:54.916323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 709/1409] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:15:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:16:03.993141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=44 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:04.915541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:09.218765 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6f852-659c-41a4-af40-3c28020f36fd None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:09.219359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6f852-659c-41a4-af40-3c28020f36fd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:09.219359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a6f852-659c-41a4-af40-3c28020f36fd None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:09.220630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 701/1410] 10.222.0.64 () {54 vars in 918 bytes} [Wed Sep 21 16:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:09.225885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27de1f-505e-4c38-ac58-43ee2943ee94 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:09.226985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27de1f-505e-4c38-ac58-43ee2943ee94 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:09.227219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a27de1f-505e-4c38-ac58-43ee2943ee94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:09.303834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0a27de1f-505e-4c38-ac58-43ee2943ee94 None None] MFA Rules not processed for user `55642294e8ff4f779510f4646e0e30a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:09.305779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0a27de1f-505e-4c38-ac58-43ee2943ee94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:09.412159 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 710/1411] 10.222.0.64 () {58 vars in 1015 bytes} [Wed Sep 21 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:09.427689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cee48-5890-4be5-ae72-371fcd7d5576 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:09.427976 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cee48-5890-4be5-ae72-371fcd7d5576 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:09.428252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cee48-5890-4be5-ae72-371fcd7d5576 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:09.429590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 702/1412] 10.222.0.64 () {54 vars in 918 bytes} [Wed Sep 21 16:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:09.435417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a91086-3dd0-40c2-8463-89f40e76bc67 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:09.435738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a91086-3dd0-40c2-8463-89f40e76bc67 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:09.436023 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a91086-3dd0-40c2-8463-89f40e76bc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:09.513469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:09.515076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-66a91086-3dd0-40c2-8463-89f40e76bc67 None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:09.516818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-66a91086-3dd0-40c2-8463-89f40e76bc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:09.633387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 711/1413] 10.222.0.64 () {58 vars in 1015 bytes} [Wed Sep 21 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:12.250338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3286a-48cc-4751-8935-7a3e51973cea None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:12.250880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3286a-48cc-4751-8935-7a3e51973cea None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:12.250880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3286a-48cc-4751-8935-7a3e51973cea None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:12.252229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:12.258810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ca9e8-5528-4104-a469-37dbf67225c5 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:12.259409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ca9e8-5528-4104-a469-37dbf67225c5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:12.259409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ca9e8-5528-4104-a469-37dbf67225c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:12.328960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6b0ca9e8-5528-4104-a469-37dbf67225c5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:12.330128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6b0ca9e8-5528-4104-a469-37dbf67225c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:12.386677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:12.430866 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:12.721783 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3286a-48cc-4751-8935-7a3e51973cea None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:12.723483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b8e3286a-48cc-4751-8935-7a3e51973cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:12.885741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3286a-48cc-4751-8935-7a3e51973cea None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:12.963649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:13.005054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed36a97-465d-4087-8771-4f4213fab62c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:13.005658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed36a97-465d-4087-8771-4f4213fab62c service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:13.005931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed36a97-465d-4087-8771-4f4213fab62c service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:13.006228 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed36a97-465d-4087-8771-4f4213fab62c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:13.008388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-eed36a97-465d-4087-8771-4f4213fab62c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:13.099241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed36a97-465d-4087-8771-4f4213fab62c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:13.100845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed36a97-465d-4087-8771-4f4213fab62c service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:13.119735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 704/1416] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:14.264090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa802f-dc49-49de-a019-b6497d549697 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:14.264643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa802f-dc49-49de-a019-b6497d549697 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:14.264643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aa802f-dc49-49de-a019-b6497d549697 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:14.265983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 16:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:14.272347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f320ddd-ef76-4b75-b0f1-8c2949c2c478 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:14.272897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f320ddd-ef76-4b75-b0f1-8c2949c2c478 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:14.272897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f320ddd-ef76-4b75-b0f1-8c2949c2c478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:14.353946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3f320ddd-ef76-4b75-b0f1-8c2949c2c478 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:14.355187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3f320ddd-ef76-4b75-b0f1-8c2949c2c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:14.383921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:14.456423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 16:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:14.607356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa802f-dc49-49de-a019-b6497d549697 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:14.608919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-44aa802f-dc49-49de-a019-b6497d549697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:14.755724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aa802f-dc49-49de-a019-b6497d549697 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:14.807297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:14.883725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bd24f-4701-41e8-bf1d-23c29e17857a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:14.884334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd24f-4701-41e8-bf1d-23c29e17857a service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:14.884619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd24f-4701-41e8-bf1d-23c29e17857a service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:14.884891 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd24f-4701-41e8-bf1d-23c29e17857a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:14.886518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-352bd24f-4701-41e8-bf1d-23c29e17857a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:14.983507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bd24f-4701-41e8-bf1d-23c29e17857a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:14.985133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352bd24f-4701-41e8-bf1d-23c29e17857a service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:15.005774 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 714/1419] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:16.166783 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3e1b8-a011-4e08-b772-9f9757f19cf4 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:16.167361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3e1b8-a011-4e08-b772-9f9757f19cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:16.167361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3e1b8-a011-4e08-b772-9f9757f19cf4 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:16.168709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 16:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:16.175097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476f76c-0492-41aa-9648-e9b3b9cec97f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:16.175544 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476f76c-0492-41aa-9648-e9b3b9cec97f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:16.175636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476f76c-0492-41aa-9648-e9b3b9cec97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:16.229546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:16.251657 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2476f76c-0492-41aa-9648-e9b3b9cec97f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:16.252875 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2476f76c-0492-41aa-9648-e9b3b9cec97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:16.351730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 16:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:16.503867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3e1b8-a011-4e08-b772-9f9757f19cf4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:16.505535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b2a3e1b8-a011-4e08-b772-9f9757f19cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:16.649084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3e1b8-a011-4e08-b772-9f9757f19cf4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:16.661470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:16.763895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3274e-6745-427e-9c38-2d11af659065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:16.764408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3274e-6745-427e-9c38-2d11af659065 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:16.764689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3274e-6745-427e-9c38-2d11af659065 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:16.764956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3274e-6745-427e-9c38-2d11af659065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:16.766578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dfd3274e-6745-427e-9c38-2d11af659065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:16.860781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3274e-6745-427e-9c38-2d11af659065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:16.862440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3274e-6745-427e-9c38-2d11af659065 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:16.887545 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 707/1422] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:18.033618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379e4b9-c4fd-4e2e-814a-2dd8cec461ab None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:18.034297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379e4b9-c4fd-4e2e-814a-2dd8cec461ab None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:18.034297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379e4b9-c4fd-4e2e-814a-2dd8cec461ab None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:18.035602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 16:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:18.042655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9da233-4b64-433b-b0aa-5187de3ca53f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:18.043140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9da233-4b64-433b-b0aa-5187de3ca53f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:18.043247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9da233-4b64-433b-b0aa-5187de3ca53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:18.063848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:18.122188 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5e9da233-4b64-433b-b0aa-5187de3ca53f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:18.123400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5e9da233-4b64-433b-b0aa-5187de3ca53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:18.223193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 16:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:18.373103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379e4b9-c4fd-4e2e-814a-2dd8cec461ab None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:18.374654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3379e4b9-c4fd-4e2e-814a-2dd8cec461ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:18.486375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:18.526583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379e4b9-c4fd-4e2e-814a-2dd8cec461ab None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:18.642217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15e534-5bfe-4746-9585-be420db0f522 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:18.642740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15e534-5bfe-4746-9585-be420db0f522 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:18.643017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15e534-5bfe-4746-9585-be420db0f522 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:18.643287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15e534-5bfe-4746-9585-be420db0f522 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:18.644877 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0c15e534-5bfe-4746-9585-be420db0f522 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:18.720885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:18.735165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c15e534-5bfe-4746-9585-be420db0f522 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:18.736780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c15e534-5bfe-4746-9585-be420db0f522 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:18.761099 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 717/1425] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:19.898797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3091b87-dc2c-487a-b268-0a63a8472791 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:19.899337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3091b87-dc2c-487a-b268-0a63a8472791 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:19.899337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3091b87-dc2c-487a-b268-0a63a8472791 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:19.900582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 16:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:19.907313 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24d455-4829-4a68-b585-3431e5049253 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:19.907826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24d455-4829-4a68-b585-3431e5049253 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:19.907937 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24d455-4829-4a68-b585-3431e5049253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:19.985547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5b24d455-4829-4a68-b585-3431e5049253 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:19.986792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5b24d455-4829-4a68-b585-3431e5049253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:20.086751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 16:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:20.238384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3091b87-dc2c-487a-b268-0a63a8472791 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:20.239948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e3091b87-dc2c-487a-b268-0a63a8472791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:20.325485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:20.388091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3091b87-dc2c-487a-b268-0a63a8472791 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:20.505039 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dd9a2-6bea-4f2c-a1de-41be52bf9086 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:20.505602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dd9a2-6bea-4f2c-a1de-41be52bf9086 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:20.505888 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dd9a2-6bea-4f2c-a1de-41be52bf9086 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:20.506194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dd9a2-6bea-4f2c-a1de-41be52bf9086 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:20.507822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-aa5dd9a2-6bea-4f2c-a1de-41be52bf9086 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:20.558871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:20.603771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5dd9a2-6bea-4f2c-a1de-41be52bf9086 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:20.605412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5dd9a2-6bea-4f2c-a1de-41be52bf9086 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:20.625037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 710/1428] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:21.700094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db79b59-3320-438b-b86c-8c31186a4938 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:21.700656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db79b59-3320-438b-b86c-8c31186a4938 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:21.700656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db79b59-3320-438b-b86c-8c31186a4938 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:21.702078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 16:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:21.708750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1714856-4a0e-44d0-9ec4-f57387eb272c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:21.709211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1714856-4a0e-44d0-9ec4-f57387eb272c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:21.709323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1714856-4a0e-44d0-9ec4-f57387eb272c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:21.788353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f1714856-4a0e-44d0-9ec4-f57387eb272c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:21.789470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f1714856-4a0e-44d0-9ec4-f57387eb272c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:21.886731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 16:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:22.045228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db79b59-3320-438b-b86c-8c31186a4938 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:22.046732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0db79b59-3320-438b-b86c-8c31186a4938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:22.098641 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:22.197259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db79b59-3320-438b-b86c-8c31186a4938 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:22.309639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964959b7-8504-434a-bc68-f3a76d39da8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:22.310188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964959b7-8504-434a-bc68-f3a76d39da8e service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:22.310470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964959b7-8504-434a-bc68-f3a76d39da8e service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:22.310742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964959b7-8504-434a-bc68-f3a76d39da8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:22.312387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-964959b7-8504-434a-bc68-f3a76d39da8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:22.320274 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:22.405526 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964959b7-8504-434a-bc68-f3a76d39da8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:22.407170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964959b7-8504-434a-bc68-f3a76d39da8e service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:22.425962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 720/1431] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:23.519899 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896873c-d2da-4985-b54f-ed4adf1af124 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:23.520466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896873c-d2da-4985-b54f-ed4adf1af124 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:23.520466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896873c-d2da-4985-b54f-ed4adf1af124 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:23.521807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 16:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:23.529697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbcfd0b-be9b-4fbb-a339-d8db8c1c5b01 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:23.530300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbcfd0b-be9b-4fbb-a339-d8db8c1c5b01 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:23.530300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbcfd0b-be9b-4fbb-a339-d8db8c1c5b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:23.614022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7fbcfd0b-be9b-4fbb-a339-d8db8c1c5b01 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:23.615284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7fbcfd0b-be9b-4fbb-a339-d8db8c1c5b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:23.684425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:23.715121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 16:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:23.864818 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896873c-d2da-4985-b54f-ed4adf1af124 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:23.866541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0896873c-d2da-4985-b54f-ed4adf1af124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:23.885833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:24.026219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896873c-d2da-4985-b54f-ed4adf1af124 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:24.124580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:24.151766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a573e8a-5e64-40dc-aa05-7889a5872d9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:24.152368 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a573e8a-5e64-40dc-aa05-7889a5872d9b service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:24.152645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a573e8a-5e64-40dc-aa05-7889a5872d9b service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:24.152918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a573e8a-5e64-40dc-aa05-7889a5872d9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:24.154558 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0a573e8a-5e64-40dc-aa05-7889a5872d9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:24.245273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a573e8a-5e64-40dc-aa05-7889a5872d9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:24.246906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a573e8a-5e64-40dc-aa05-7889a5872d9b service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:24.265451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 713/1434] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:16:25.384352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830e26d-2475-4525-a9f5-0a52ba1994f3 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:25.384997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830e26d-2475-4525-a9f5-0a52ba1994f3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:25.384997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830e26d-2475-4525-a9f5-0a52ba1994f3 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:25.386344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 16:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:25.393549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2a033-024e-44d0-9345-e165732bdb23 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:25.394160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2a033-024e-44d0-9345-e165732bdb23 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:25.394160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2a033-024e-44d0-9345-e165732bdb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:25.604356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b8f2a033-024e-44d0-9345-e165732bdb23 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:25.605588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b8f2a033-024e-44d0-9345-e165732bdb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:25.648577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:25.705921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 16:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:16:25.857102 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830e26d-2475-4525-a9f5-0a52ba1994f3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:25.858739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7830e26d-2475-4525-a9f5-0a52ba1994f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:26.007452 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830e26d-2475-4525-a9f5-0a52ba1994f3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:26.062852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:26.124473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f75478-77bd-4d43-8344-8338c452c806 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:26.125083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f75478-77bd-4d43-8344-8338c452c806 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:26.125363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f75478-77bd-4d43-8344-8338c452c806 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:26.125644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f75478-77bd-4d43-8344-8338c452c806 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:26.127310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c7f75478-77bd-4d43-8344-8338c452c806 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:26.217111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f75478-77bd-4d43-8344-8338c452c806 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:26.218764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f75478-77bd-4d43-8344-8338c452c806 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:26.237098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 723/1437] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:27.311314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8207e-3195-44a6-82ad-d2b2119dee54 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:27.311912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8207e-3195-44a6-82ad-d2b2119dee54 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:27.311912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8207e-3195-44a6-82ad-d2b2119dee54 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:27.313192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 16:16:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:27.319750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8833b4f-8711-4f31-ba92-73fa0830bb4d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:27.320241 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8833b4f-8711-4f31-ba92-73fa0830bb4d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:27.320338 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8833b4f-8711-4f31-ba92-73fa0830bb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:27.402408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a8833b4f-8711-4f31-ba92-73fa0830bb4d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:27.403623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a8833b4f-8711-4f31-ba92-73fa0830bb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:27.405651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:27.504897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 16:16:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:27.656256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8207e-3195-44a6-82ad-d2b2119dee54 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:27.657789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a0a8207e-3195-44a6-82ad-d2b2119dee54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:27.803431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8207e-3195-44a6-82ad-d2b2119dee54 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:27.830575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:27.918308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289907f7-1cb0-47de-a72d-aae4c90854a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:27.918828 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289907f7-1cb0-47de-a72d-aae4c90854a8 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:27.919104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289907f7-1cb0-47de-a72d-aae4c90854a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:27.919370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289907f7-1cb0-47de-a72d-aae4c90854a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:27.920965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-289907f7-1cb0-47de-a72d-aae4c90854a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:28.010997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289907f7-1cb0-47de-a72d-aae4c90854a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:28.012620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289907f7-1cb0-47de-a72d-aae4c90854a8 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:28.031059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 716/1440] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:30.122321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772aa2c-da5a-40e5-947d-a0e50a99d78e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:30.122935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772aa2c-da5a-40e5-947d-a0e50a99d78e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:30.122935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772aa2c-da5a-40e5-947d-a0e50a99d78e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:30.124203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:30.129595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a887a-4906-4bb0-a36c-ec66f102b415 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:30.130072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a887a-4906-4bb0-a36c-ec66f102b415 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:30.130198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9a887a-4906-4bb0-a36c-ec66f102b415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:30.173169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:30.200249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bd9a887a-4906-4bb0-a36c-ec66f102b415 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:30.201462 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bd9a887a-4906-4bb0-a36c-ec66f102b415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:30.296154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:30.578685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772aa2c-da5a-40e5-947d-a0e50a99d78e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:30.580230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9772aa2c-da5a-40e5-947d-a0e50a99d78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:30.701744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772aa2c-da5a-40e5-947d-a0e50a99d78e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:30.735570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:30.779503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646c930b-85f6-4568-9ef9-8875aeb93255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:30.780024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c930b-85f6-4568-9ef9-8875aeb93255 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:30.780298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c930b-85f6-4568-9ef9-8875aeb93255 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:30.780568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646c930b-85f6-4568-9ef9-8875aeb93255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:30.782265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-646c930b-85f6-4568-9ef9-8875aeb93255 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:30.903005 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c930b-85f6-4568-9ef9-8875aeb93255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:30.904445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646c930b-85f6-4568-9ef9-8875aeb93255 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:30.931891 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 726/1443] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:33.295757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048350df-b491-49b1-8137-5111440f8ac2 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:33.296399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048350df-b491-49b1-8137-5111440f8ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:33.296399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048350df-b491-49b1-8137-5111440f8ac2 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:33.297673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:33.303469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4540e-a065-41a1-bd9f-2fafca7f3828 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:33.304254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4540e-a065-41a1-bd9f-2fafca7f3828 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:33.304362 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4540e-a065-41a1-bd9f-2fafca7f3828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:33.373335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ade4540e-a065-41a1-bd9f-2fafca7f3828 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:33.374602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ade4540e-a065-41a1-bd9f-2fafca7f3828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:33.387627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:33.480600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:33.774133 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048350df-b491-49b1-8137-5111440f8ac2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:33.776555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-048350df-b491-49b1-8137-5111440f8ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:33.884217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048350df-b491-49b1-8137-5111440f8ac2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:33.900544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:33.977908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be35b2-80f2-41de-9638-428d8d0b0bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:33.978557 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be35b2-80f2-41de-9638-428d8d0b0bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:33.978836 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be35b2-80f2-41de-9638-428d8d0b0bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:33.979106 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be35b2-80f2-41de-9638-428d8d0b0bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:33.980767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-08be35b2-80f2-41de-9638-428d8d0b0bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:34.096300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be35b2-80f2-41de-9638-428d8d0b0bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:34.097655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be35b2-80f2-41de-9638-428d8d0b0bd0 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:34.116304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 719/1446] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:36.623159 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea282351-d0d5-4547-9650-df747e849d4e None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:36.623810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea282351-d0d5-4547-9650-df747e849d4e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:36.623810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea282351-d0d5-4547-9650-df747e849d4e None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:36.625052 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:36.631841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aaafd6-3074-4039-849d-78c4b98c65cc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:36.632335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aaafd6-3074-4039-849d-78c4b98c65cc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:36.632430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aaafd6-3074-4039-849d-78c4b98c65cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:36.682313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:36.702451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f7aaafd6-3074-4039-849d-78c4b98c65cc None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:36.703747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f7aaafd6-3074-4039-849d-78c4b98c65cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:36.805112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:37.093847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea282351-d0d5-4547-9650-df747e849d4e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:37.095425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ea282351-d0d5-4547-9650-df747e849d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:37.201761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea282351-d0d5-4547-9650-df747e849d4e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:37.246396 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:37.300414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa490a-9726-42e6-8e54-3ae187292234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:37.301047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa490a-9726-42e6-8e54-3ae187292234 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:37.301328 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa490a-9726-42e6-8e54-3ae187292234 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:37.301599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa490a-9726-42e6-8e54-3ae187292234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:37.304162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-acfa490a-9726-42e6-8e54-3ae187292234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:37.436379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa490a-9726-42e6-8e54-3ae187292234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:37.437838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa490a-9726-42e6-8e54-3ae187292234 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:37.457113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 729/1449] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:39.783173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6a38d-dceb-4ebc-8e2a-73c89675b162 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:39.783696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6a38d-dceb-4ebc-8e2a-73c89675b162 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:39.783696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6a38d-dceb-4ebc-8e2a-73c89675b162 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:39.784978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:39.790697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67703e-6f77-42b8-9575-00da3dcaffff None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:39.791117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67703e-6f77-42b8-9575-00da3dcaffff None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:39.791234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67703e-6f77-42b8-9575-00da3dcaffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:39.860721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3b67703e-6f77-42b8-9575-00da3dcaffff None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:39.861932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3b67703e-6f77-42b8-9575-00da3dcaffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:39.882939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:39.961429 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:40.245140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6a38d-dceb-4ebc-8e2a-73c89675b162 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:40.246687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c7a6a38d-dceb-4ebc-8e2a-73c89675b162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:40.353599 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6a38d-dceb-4ebc-8e2a-73c89675b162 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:40.374386 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:40.433468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a905fdb-8a0d-49bf-9cf3-20edb9a6dcf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:40.434112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a905fdb-8a0d-49bf-9cf3-20edb9a6dcf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:40.434396 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a905fdb-8a0d-49bf-9cf3-20edb9a6dcf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:40.434667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a905fdb-8a0d-49bf-9cf3-20edb9a6dcf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:40.436383 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4a905fdb-8a0d-49bf-9cf3-20edb9a6dcf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:40.561088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a905fdb-8a0d-49bf-9cf3-20edb9a6dcf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:40.562482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a905fdb-8a0d-49bf-9cf3-20edb9a6dcf4 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:40.581925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 722/1452] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:42.812747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f218d9-0ee8-40c5-9f99-2132f201d344 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:42.813285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f218d9-0ee8-40c5-9f99-2132f201d344 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:42.813285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f218d9-0ee8-40c5-9f99-2132f201d344 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:42.814655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:42.822010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5a60a-0a62-4ed9-b82b-00ca71236e41 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:42.822523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5a60a-0a62-4ed9-b82b-00ca71236e41 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:42.822618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5a60a-0a62-4ed9-b82b-00ca71236e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:42.887232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:42.890031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0aa5a60a-0a62-4ed9-b82b-00ca71236e41 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:42.891863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0aa5a60a-0a62-4ed9-b82b-00ca71236e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:42.988637 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:43.269787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f218d9-0ee8-40c5-9f99-2132f201d344 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:43.271941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-54f218d9-0ee8-40c5-9f99-2132f201d344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:43.380364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f218d9-0ee8-40c5-9f99-2132f201d344 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:43.433317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:43.459675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2785e-398e-4dbb-8d9c-66d209b1eb21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:43.460287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2785e-398e-4dbb-8d9c-66d209b1eb21 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:43.460564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2785e-398e-4dbb-8d9c-66d209b1eb21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:43.460831 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2785e-398e-4dbb-8d9c-66d209b1eb21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:43.462484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-24e2785e-398e-4dbb-8d9c-66d209b1eb21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:43.580738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2785e-398e-4dbb-8d9c-66d209b1eb21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:43.582172 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2785e-398e-4dbb-8d9c-66d209b1eb21 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:43.602838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 732/1455] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:45.868209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a98f4a-4155-4f94-8317-9f87c65d53bd None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:45.868736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a98f4a-4155-4f94-8317-9f87c65d53bd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:45.868736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a98f4a-4155-4f94-8317-9f87c65d53bd None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:45.870201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:45.875620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0b726-4766-4216-bf87-ea67c694d263 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:45.876038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0b726-4766-4216-bf87-ea67c694d263 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:45.876172 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe0b726-4766-4216-bf87-ea67c694d263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:45.947647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ebe0b726-4766-4216-bf87-ea67c694d263 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:45.948853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ebe0b726-4766-4216-bf87-ea67c694d263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:45.978353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:46.045505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:46.328138 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a98f4a-4155-4f94-8317-9f87c65d53bd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:46.329722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-85a98f4a-4155-4f94-8317-9f87c65d53bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:46.445419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a98f4a-4155-4f94-8317-9f87c65d53bd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:46.475039 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:46.524998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b5f42-1e6e-44b7-9de9-d1bedc5d3288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:46.525599 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b5f42-1e6e-44b7-9de9-d1bedc5d3288 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:46.525872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b5f42-1e6e-44b7-9de9-d1bedc5d3288 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:46.526169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b5f42-1e6e-44b7-9de9-d1bedc5d3288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:46.527760 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-291b5f42-1e6e-44b7-9de9-d1bedc5d3288 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:46.638518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b5f42-1e6e-44b7-9de9-d1bedc5d3288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:46.639972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b5f42-1e6e-44b7-9de9-d1bedc5d3288 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:46.663673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 725/1458] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:48.904191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757f82e-2e25-4069-8381-ed77969931e0 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:48.904793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757f82e-2e25-4069-8381-ed77969931e0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:48.904793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757f82e-2e25-4069-8381-ed77969931e0 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:48.906069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:48.911694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782dd1e-7ff1-4a85-b333-cf2e70227180 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:48.912238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782dd1e-7ff1-4a85-b333-cf2e70227180 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:48.912366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782dd1e-7ff1-4a85-b333-cf2e70227180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:48.981203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1782dd1e-7ff1-4a85-b333-cf2e70227180 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:48.982457 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1782dd1e-7ff1-4a85-b333-cf2e70227180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:48.991646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:49.092625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:49.377148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757f82e-2e25-4069-8381-ed77969931e0 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:49.378780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8757f82e-2e25-4069-8381-ed77969931e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:49.488407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757f82e-2e25-4069-8381-ed77969931e0 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:49.547384 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:49.566421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e42f0-10df-40ab-9c15-1a8a539fd00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:49.566715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e42f0-10df-40ab-9c15-1a8a539fd00e service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:49.567546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e42f0-10df-40ab-9c15-1a8a539fd00e service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:49.567928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e42f0-10df-40ab-9c15-1a8a539fd00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:49.569438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d61e42f0-10df-40ab-9c15-1a8a539fd00e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:49.679439 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e42f0-10df-40ab-9c15-1a8a539fd00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:49.681200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e42f0-10df-40ab-9c15-1a8a539fd00e service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:49.699908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 735/1461] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:51.921716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaddaca-8853-473c-89a3-c6185d94151f None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:51.922318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaddaca-8853-473c-89a3-c6185d94151f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:51.922318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaddaca-8853-473c-89a3-c6185d94151f None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:51.923662 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:51.931035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9f39e-82d9-41b5-8282-5f5c249ba024 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:51.931602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9f39e-82d9-41b5-8282-5f5c249ba024 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:51.931714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9f39e-82d9-41b5-8282-5f5c249ba024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:52.011108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-09d9f39e-82d9-41b5-8282-5f5c249ba024 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:52.012341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-09d9f39e-82d9-41b5-8282-5f5c249ba024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:52.054374 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:52.114853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:52.396169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaddaca-8853-473c-89a3-c6185d94151f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:52.397708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-beaddaca-8853-473c-89a3-c6185d94151f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:52.505319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaddaca-8853-473c-89a3-c6185d94151f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:52.545721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:52.583052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13b251c-70da-4f5f-8906-c9e20e997c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:52.583656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b251c-70da-4f5f-8906-c9e20e997c14 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:52.583932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b251c-70da-4f5f-8906-c9e20e997c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:52.584204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13b251c-70da-4f5f-8906-c9e20e997c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:52.585822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b13b251c-70da-4f5f-8906-c9e20e997c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:52.701473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b251c-70da-4f5f-8906-c9e20e997c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:52.703080 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13b251c-70da-4f5f-8906-c9e20e997c14 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:52.722576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 728/1464] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:54.919680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df279547-02dc-4393-bd20-c5e2785ff492 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:54.920271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df279547-02dc-4393-bd20-c5e2785ff492 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:54.920271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df279547-02dc-4393-bd20-c5e2785ff492 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:54.921571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:54.927450 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd159b-c290-4b90-8172-bd32f7ae76fd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:54.928229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd159b-c290-4b90-8172-bd32f7ae76fd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:54.928334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd159b-c290-4b90-8172-bd32f7ae76fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:54.996964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-62bd159b-c290-4b90-8172-bd32f7ae76fd None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:54.998222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-62bd159b-c290-4b90-8172-bd32f7ae76fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:55.019838 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:55.101946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:55.383080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df279547-02dc-4393-bd20-c5e2785ff492 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:55.384686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-df279547-02dc-4393-bd20-c5e2785ff492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:55.500535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df279547-02dc-4393-bd20-c5e2785ff492 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:55.571814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:55.579267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed76772-4eea-4b27-b604-d2f1e3149b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:55.579875 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed76772-4eea-4b27-b604-d2f1e3149b24 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:55.580198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed76772-4eea-4b27-b604-d2f1e3149b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:55.580467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed76772-4eea-4b27-b604-d2f1e3149b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:55.582159 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3ed76772-4eea-4b27-b604-d2f1e3149b24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:55.694150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed76772-4eea-4b27-b604-d2f1e3149b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:55.695592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed76772-4eea-4b27-b604-d2f1e3149b24 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:55.714710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 738/1467] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:16:57.919276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b016b-253c-405a-85a6-e7490ae9c763 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:57.919839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b016b-253c-405a-85a6-e7490ae9c763 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:57.919839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b016b-253c-405a-85a6-e7490ae9c763 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:57.921155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:16:57.926910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eae4a4-474d-4de8-a8d6-6a4e1371155d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:57.927345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eae4a4-474d-4de8-a8d6-6a4e1371155d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:57.927446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eae4a4-474d-4de8-a8d6-6a4e1371155d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:57.999872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-02eae4a4-474d-4de8-a8d6-6a4e1371155d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:16:58.001101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-02eae4a4-474d-4de8-a8d6-6a4e1371155d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:58.057690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:58.112927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:16:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:16:58.396361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b016b-253c-405a-85a6-e7490ae9c763 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:16:58.397913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2c1b016b-253c-405a-85a6-e7490ae9c763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:58.505722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b016b-253c-405a-85a6-e7490ae9c763 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:16:58.557427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:16:58.585343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aa8cca-53ad-41d9-ac7e-d842eecf7470 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:16:58.585940 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa8cca-53ad-41d9-ac7e-d842eecf7470 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:16:58.586246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa8cca-53ad-41d9-ac7e-d842eecf7470 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:16:58.586517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aa8cca-53ad-41d9-ac7e-d842eecf7470 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:16:58.588139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b5aa8cca-53ad-41d9-ac7e-d842eecf7470 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:16:58.701330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa8cca-53ad-41d9-ac7e-d842eecf7470 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:16:58.703115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aa8cca-53ad-41d9-ac7e-d842eecf7470 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:16:58.722541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 731/1470] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:17:08.110625 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=22 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:08.720058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=66 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:19.493920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee46d8-1cc6-4814-ba44-54b43f4b9ce8 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:19.494511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee46d8-1cc6-4814-ba44-54b43f4b9ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:19.494511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee46d8-1cc6-4814-ba44-54b43f4b9ce8 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:19.495817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 740/1471] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:17:19.501065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55654f3c-6fb1-4285-b90f-90cafe797e03 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:19.501511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55654f3c-6fb1-4285-b90f-90cafe797e03 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:19.501693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55654f3c-6fb1-4285-b90f-90cafe797e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:19.567910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-55654f3c-6fb1-4285-b90f-90cafe797e03 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:17:19.569111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-55654f3c-6fb1-4285-b90f-90cafe797e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:19.677664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 732/1472] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:17:19.978462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646d1b4-f34e-4805-87d2-62146c175151 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:19.978746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646d1b4-f34e-4805-87d2-62146c175151 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:19.979017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646d1b4-f34e-4805-87d2-62146c175151 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:19.980343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 741/1473] 10.222.0.64 () {54 vars in 962 bytes} [Wed Sep 21 16:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:17:19.985346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032a5e3-e22c-46a8-9f57-a858cf457e50 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:19.985640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032a5e3-e22c-46a8-9f57-a858cf457e50 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:19.985905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032a5e3-e22c-46a8-9f57-a858cf457e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:20.064359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7032a5e3-e22c-46a8-9f57-a858cf457e50 None None] MFA Rules not processed for user `d2d76fb9cbdd40f9b2a748209cee5b75`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:17:20.066174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7032a5e3-e22c-46a8-9f57-a858cf457e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:20.087413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:20.146264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 733/1474] 10.222.0.64 () {58 vars in 1059 bytes} [Wed Sep 21 16:17:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:17:20.154138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646d1b4-f34e-4805-87d2-62146c175151 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:17:20.155663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e646d1b4-f34e-4805-87d2-62146c175151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:20.260936 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646d1b4-f34e-4805-87d2-62146c175151 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:17:20.340234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7556e-a033-43bd-b200-c76dba1f93d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2d76fb9cbdd40f9b2a748209cee5b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:17:20.340839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7556e-a033-43bd-b200-c76dba1f93d9 service glance] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:20.341117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7556e-a033-43bd-b200-c76dba1f93d9 service glance] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:20.341389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7556e-a033-43bd-b200-c76dba1f93d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:20.343007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1ff7556e-a033-43bd-b200-c76dba1f93d9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:20.398425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:20.456793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7556e-a033-43bd-b200-c76dba1f93d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:17:20.458267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7556e-a033-43bd-b200-c76dba1f93d9 service glance] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:17:20.476961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 742/1475] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:17:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:17:22.747668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccfb71-39c2-4bd7-a6bc-1499461c9459 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:22.748232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccfb71-39c2-4bd7-a6bc-1499461c9459 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:22.748232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ccfb71-39c2-4bd7-a6bc-1499461c9459 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:22.749461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 734/1476] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:17:22.755466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9834c-6da4-4594-8b9a-4d8a52b2da54 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:22.755994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9834c-6da4-4594-8b9a-4d8a52b2da54 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:22.755994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9834c-6da4-4594-8b9a-4d8a52b2da54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:22.825724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ddf9834c-6da4-4594-8b9a-4d8a52b2da54 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:17:22.826999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ddf9834c-6da4-4594-8b9a-4d8a52b2da54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:22.928299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 743/1477] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:17:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:17:23.206646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccfb71-39c2-4bd7-a6bc-1499461c9459 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:17:23.208329 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-38ccfb71-39c2-4bd7-a6bc-1499461c9459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:23.348430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ccfb71-39c2-4bd7-a6bc-1499461c9459 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:17:23.399829 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:23.449007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6c2a87-8216-4533-9ba9-0425d4483c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:17:23.449610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c2a87-8216-4533-9ba9-0425d4483c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:23.449885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c2a87-8216-4533-9ba9-0425d4483c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:23.450189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c2a87-8216-4533-9ba9-0425d4483c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:23.450827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c2a87-8216-4533-9ba9-0425d4483c6a admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:17:23.452568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-6a6c2a87-8216-4533-9ba9-0425d4483c6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:17:23.452568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 21 16:17:23.452568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: Sep 21 16:17:23.453697 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6c2a87-8216-4533-9ba9-0425d4483c6a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:17:23.474387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 735/1478] 10.222.0.64 () {56 vars in 1108 bytes} [Wed Sep 21 16:17:23 2022] GET /identity/v3/projects => generated 2553 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:17:25.675753 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15bf31-05a3-4b52-9ade-4a4598b27a97 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:25.676326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15bf31-05a3-4b52-9ade-4a4598b27a97 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:25.676326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15bf31-05a3-4b52-9ade-4a4598b27a97 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:25.677902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 744/1479] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:17:25.683482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307276d-51e1-425c-a2f1-3201782f9521 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:25.683922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307276d-51e1-425c-a2f1-3201782f9521 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:25.684045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307276d-51e1-425c-a2f1-3201782f9521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:25.749931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2307276d-51e1-425c-a2f1-3201782f9521 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:17:25.751082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2307276d-51e1-425c-a2f1-3201782f9521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:25.854409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 736/1480] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:17:26.151845 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15bf31-05a3-4b52-9ade-4a4598b27a97 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:17:26.153563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0c15bf31-05a3-4b52-9ade-4a4598b27a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:26.163958 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:26.302643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15bf31-05a3-4b52-9ade-4a4598b27a97 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:17:26.394148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:26.428116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0ead1-1b01-4ecf-b6d4-4d9c7a5199ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:17:26.428727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0ead1-1b01-4ecf-b6d4-4d9c7a5199ed service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:26.429005 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0ead1-1b01-4ecf-b6d4-4d9c7a5199ed service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:26.429281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0ead1-1b01-4ecf-b6d4-4d9c7a5199ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:26.430939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e6e0ead1-1b01-4ecf-b6d4-4d9c7a5199ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:26.523898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0ead1-1b01-4ecf-b6d4-4d9c7a5199ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:17:26.525518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0ead1-1b01-4ecf-b6d4-4d9c7a5199ed service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:17:26.545138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 745/1481] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:17:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:17:28.909378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c197e-36df-475c-9613-2b9e986d9947 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:28.909981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c197e-36df-475c-9613-2b9e986d9947 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:28.910245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c197e-36df-475c-9613-2b9e986d9947 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:28.912849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 737/1482] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:17:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:17:28.919862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bd0ab-353f-472c-b068-a4b18b2d9805 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:28.921186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bd0ab-353f-472c-b068-a4b18b2d9805 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:28.921186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bd0ab-353f-472c-b068-a4b18b2d9805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:28.991918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-933bd0ab-353f-472c-b068-a4b18b2d9805 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:17:28.993125 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-933bd0ab-353f-472c-b068-a4b18b2d9805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:29.039769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:29.094273 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 746/1483] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:17:29.379819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c197e-36df-475c-9613-2b9e986d9947 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:17:29.381451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-577c197e-36df-475c-9613-2b9e986d9947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:29.409639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:29.493093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c197e-36df-475c-9613-2b9e986d9947 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:17:29.571510 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a1e0fc-6ee2-4dac-b853-9ad7a96a60ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:17:29.572120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1e0fc-6ee2-4dac-b853-9ad7a96a60ac service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:29.572397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1e0fc-6ee2-4dac-b853-9ad7a96a60ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:29.572665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a1e0fc-6ee2-4dac-b853-9ad7a96a60ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:29.574298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-86a1e0fc-6ee2-4dac-b853-9ad7a96a60ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:29.650092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:29.685717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1e0fc-6ee2-4dac-b853-9ad7a96a60ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:17:29.687194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a1e0fc-6ee2-4dac-b853-9ad7a96a60ac service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:17:29.706066 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 738/1484] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:17:31.950591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0ddfc-f2e1-4556-ab1e-32760cbccd55 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:31.951183 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0ddfc-f2e1-4556-ab1e-32760cbccd55 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:31.951183 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0ddfc-f2e1-4556-ab1e-32760cbccd55 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:31.952468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 747/1485] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:17:31.958021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2c89b-4c24-41a0-b8a0-0f8a1fbb8f9a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:31.958497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2c89b-4c24-41a0-b8a0-0f8a1fbb8f9a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:31.958599 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2c89b-4c24-41a0-b8a0-0f8a1fbb8f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:32.026937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-41d2c89b-4c24-41a0-b8a0-0f8a1fbb8f9a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:17:32.028169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-41d2c89b-4c24-41a0-b8a0-0f8a1fbb8f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:32.129193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 739/1486] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:17:32.420738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0ddfc-f2e1-4556-ab1e-32760cbccd55 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:17:32.422356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-76f0ddfc-f2e1-4556-ab1e-32760cbccd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:32.528631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0ddfc-f2e1-4556-ab1e-32760cbccd55 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:17:32.602314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:32.603933 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f00a761-29c8-4dce-b58c-2da115e252fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:17:32.604648 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00a761-29c8-4dce-b58c-2da115e252fb service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:32.605113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00a761-29c8-4dce-b58c-2da115e252fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:32.605615 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f00a761-29c8-4dce-b58c-2da115e252fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:32.607637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2f00a761-29c8-4dce-b58c-2da115e252fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:32.723196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00a761-29c8-4dce-b58c-2da115e252fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:17:32.724640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f00a761-29c8-4dce-b58c-2da115e252fb service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:17:32.742827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 748/1487] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:17:35.037931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23535e97-784d-486d-9df0-c717fea29278 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:35.038607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23535e97-784d-486d-9df0-c717fea29278 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:35.038607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23535e97-784d-486d-9df0-c717fea29278 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:35.039849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 740/1488] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:17:35.045423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400adc6-aaf8-4c6f-b657-934f9bc37793 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:35.045956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400adc6-aaf8-4c6f-b657-934f9bc37793 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:35.045956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400adc6-aaf8-4c6f-b657-934f9bc37793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:35.117114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7400adc6-aaf8-4c6f-b657-934f9bc37793 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:17:35.118388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7400adc6-aaf8-4c6f-b657-934f9bc37793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:35.174934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:35.222261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 749/1489] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:17:35.524132 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23535e97-784d-486d-9df0-c717fea29278 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:17:35.525715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-23535e97-784d-486d-9df0-c717fea29278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:35.565583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:35.636286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23535e97-784d-486d-9df0-c717fea29278 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:17:35.716406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eab5b6-a232-4c64-87d6-7cd0e4746301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:17:35.717009 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eab5b6-a232-4c64-87d6-7cd0e4746301 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:35.717282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eab5b6-a232-4c64-87d6-7cd0e4746301 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:35.717546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eab5b6-a232-4c64-87d6-7cd0e4746301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:35.719184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b4eab5b6-a232-4c64-87d6-7cd0e4746301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:35.801433 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:35.838541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eab5b6-a232-4c64-87d6-7cd0e4746301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:17:35.839975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eab5b6-a232-4c64-87d6-7cd0e4746301 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:17:35.859073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 741/1490] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:17:38.320974 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7114b4-1e47-427c-877f-22797f2caff6 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:38.321507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7114b4-1e47-427c-877f-22797f2caff6 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:38.321507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7114b4-1e47-427c-877f-22797f2caff6 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:38.322821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 750/1491] 10.222.0.64 () {54 vars in 927 bytes} [Wed Sep 21 16:17:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:17:38.328467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323366e-f86c-4764-9211-85ffe4bf233e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:38.334576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323366e-f86c-4764-9211-85ffe4bf233e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:38.334576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b323366e-f86c-4764-9211-85ffe4bf233e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:38.397662 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b323366e-f86c-4764-9211-85ffe4bf233e None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:17:38.398908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b323366e-f86c-4764-9211-85ffe4bf233e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:38.500247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 742/1492] 10.222.0.64 () {58 vars in 1024 bytes} [Wed Sep 21 16:17:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:17:38.794300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7114b4-1e47-427c-877f-22797f2caff6 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:17:38.795922 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5e7114b4-1e47-427c-877f-22797f2caff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:38.923661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7114b4-1e47-427c-877f-22797f2caff6 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:17:39.119200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef2797-69bd-4d1b-aba0-8c0d5bdf2c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:17:39.119717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef2797-69bd-4d1b-aba0-8c0d5bdf2c7c service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:17:39.119997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef2797-69bd-4d1b-aba0-8c0d5bdf2c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:17:39.120266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef2797-69bd-4d1b-aba0-8c0d5bdf2c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:17:39.121863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f1ef2797-69bd-4d1b-aba0-8c0d5bdf2c7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:17:39.130906 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:39.237106 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef2797-69bd-4d1b-aba0-8c0d5bdf2c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:17:39.238580 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef2797-69bd-4d1b-aba0-8c0d5bdf2c7c service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:17:39.257260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 751/1493] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:17:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:17:48.498339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=82 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:17:49.254946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=50 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:19:27.100021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09ea88-a1a5-42a7-b34b-0eb82d7e2f46 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:27.100700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09ea88-a1a5-42a7-b34b-0eb82d7e2f46 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:27.100700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a09ea88-a1a5-42a7-b34b-0eb82d7e2f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:27.191600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9a09ea88-a1a5-42a7-b34b-0eb82d7e2f46 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:19:27.192871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9a09ea88-a1a5-42a7-b34b-0eb82d7e2f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:19:27.302913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 743/1494] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:19:27.311112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc930d6-9dd1-4854-91eb-c8b1bc7ef871 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:27.311669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc930d6-9dd1-4854-91eb-c8b1bc7ef871 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:27.311669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc930d6-9dd1-4854-91eb-c8b1bc7ef871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:27.331171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-fcc930d6-9dd1-4854-91eb-c8b1bc7ef871 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:19:27.332434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fcc930d6-9dd1-4854-91eb-c8b1bc7ef871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:19:27.341812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 752/1495] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:19:27.346742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09ea88-a1a5-42a7-b34b-0eb82d7e2f46 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:19:27.357403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a09ea88-a1a5-42a7-b34b-0eb82d7e2f46 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:19:27.362272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd2a2-f4bc-4260-b42b-0623f8e72a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:19:27.362878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd2a2-f4bc-4260-b42b-0623f8e72a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:27.363155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd2a2-f4bc-4260-b42b-0623f8e72a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:27.363467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cd2a2-f4bc-4260-b42b-0623f8e72a01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:27.364078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cd2a2-f4bc-4260-b42b-0623f8e72a01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:19:27.364647 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cd2a2-f4bc-4260-b42b-0623f8e72a01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:19:27.365382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-e14cd2a2-f4bc-4260-b42b-0623f8e72a01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:19:27.365382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 21 16:19:27.365382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: Sep 21 16:19:27.366292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cd2a2-f4bc-4260-b42b-0623f8e72a01 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:19:27.378343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 744/1496] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:19:27.386724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc930d6-9dd1-4854-91eb-c8b1bc7ef871 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:19:27.395754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc930d6-9dd1-4854-91eb-c8b1bc7ef871 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:19:27.400365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665901a-5b8c-4057-af43-9b3019d58aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:19:27.401542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665901a-5b8c-4057-af43-9b3019d58aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:27.401819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665901a-5b8c-4057-af43-9b3019d58aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:27.402116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665901a-5b8c-4057-af43-9b3019d58aa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:27.402744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665901a-5b8c-4057-af43-9b3019d58aa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:19:27.404504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-4665901a-5b8c-4057-af43-9b3019d58aa4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:19:27.404504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 21 16:19:27.404504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: Sep 21 16:19:27.405682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665901a-5b8c-4057-af43-9b3019d58aa4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:19:27.448509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 753/1497] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:19:27 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:19:27.452764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd2a2-f4bc-4260-b42b-0623f8e72a01 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:19:27.463272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cd2a2-f4bc-4260-b42b-0623f8e72a01 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:19:27.516249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:19:27.524073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8ac23-c573-4bff-a70e-9b26704a98ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:19:27.524686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8ac23-c573-4bff-a70e-9b26704a98ec admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:27.524965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8ac23-c573-4bff-a70e-9b26704a98ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:27.525239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8ac23-c573-4bff-a70e-9b26704a98ec admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:27.526420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8ac23-c573-4bff-a70e-9b26704a98ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:19:27.527431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-62f8ac23-c573-4bff-a70e-9b26704a98ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:19:27.527431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 21 16:19:27.527431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: Sep 21 16:19:27.528681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8ac23-c573-4bff-a70e-9b26704a98ec admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:19:27.551792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 745/1498] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:19:27 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:19:27.556132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665901a-5b8c-4057-af43-9b3019d58aa4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:19:27.566662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665901a-5b8c-4057-af43-9b3019d58aa4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:19:27.571607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a324246-c0da-47ae-b988-05d0d0375fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:19:27.572089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a324246-c0da-47ae-b988-05d0d0375fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:27.573373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a324246-c0da-47ae-b988-05d0d0375fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:27.573373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a324246-c0da-47ae-b988-05d0d0375fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:27.573805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a324246-c0da-47ae-b988-05d0d0375fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:19:27.574854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-5a324246-c0da-47ae-b988-05d0d0375fe8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:19:27.574854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 21 16:19:27.574854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: Sep 21 16:19:27.575729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a324246-c0da-47ae-b988-05d0d0375fe8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:19:27.590282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 754/1499] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:19:27.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8ac23-c573-4bff-a70e-9b26704a98ec admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:19:27.604717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8ac23-c573-4bff-a70e-9b26704a98ec admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:19:27.609398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4554ec79-7be4-4c25-977d-362bae0f60eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:19:27.610004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554ec79-7be4-4c25-977d-362bae0f60eb admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:27.610321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554ec79-7be4-4c25-977d-362bae0f60eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:27.610595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4554ec79-7be4-4c25-977d-362bae0f60eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:27.611232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554ec79-7be4-4c25-977d-362bae0f60eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:19:27.612247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-4554ec79-7be4-4c25-977d-362bae0f60eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:19:27.612247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 21 16:19:27.612247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: Sep 21 16:19:27.613130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4554ec79-7be4-4c25-977d-362bae0f60eb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:19:27.627064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 746/1500] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:19:27.631392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a324246-c0da-47ae-b988-05d0d0375fe8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:19:27.642395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a324246-c0da-47ae-b988-05d0d0375fe8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:19:27.646997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d24839-51d2-4c3d-9dbf-cc03c2322091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:19:27.647657 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d24839-51d2-4c3d-9dbf-cc03c2322091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:27.647935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d24839-51d2-4c3d-9dbf-cc03c2322091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:27.648209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d24839-51d2-4c3d-9dbf-cc03c2322091 admin admin] PATH_INFO: `/v3/projects/e4fad3f2529048dfa6bca0519627a052/users/29ad6b0e18f74f0f817c04afbab7a605/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:27.663695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d24839-51d2-4c3d-9dbf-cc03c2322091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4fad3f2529048dfa6bca0519627a052, user_id=29ad6b0e18f74f0f817c04afbab7a605, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:19:27.664752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-86d24839-51d2-4c3d-9dbf-cc03c2322091 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:19:27.664752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 21 16:19:27.664752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: Sep 21 16:19:27.666732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d24839-51d2-4c3d-9dbf-cc03c2322091 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:19:27.680358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 755/1501] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:19:27 2022] PUT /identity/v3/projects/e4fad3f2529048dfa6bca0519627a052/users/29ad6b0e18f74f0f817c04afbab7a605/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:19:27.692610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34776d-56d0-4ec1-9d33-6b2726b3a183 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:27.692895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34776d-56d0-4ec1-9d33-6b2726b3a183 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:27.693171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34776d-56d0-4ec1-9d33-6b2726b3a183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:27.722443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4c34776d-56d0-4ec1-9d33-6b2726b3a183 None None] MFA Rules not processed for user `29ad6b0e18f74f0f817c04afbab7a605`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:19:27.723657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4c34776d-56d0-4ec1-9d33-6b2726b3a183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:19:27.801850 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 747/1502] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:19:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:19:27.815030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220f457-9c97-48b8-83d7-07e3c6937670 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:27.815318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220f457-9c97-48b8-83d7-07e3c6937670 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:27.815595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f220f457-9c97-48b8-83d7-07e3c6937670 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:27.816929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 756/1503] 10.222.0.64 () {54 vars in 963 bytes} [Wed Sep 21 16:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:19:27.821718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb7190-d786-48e7-a290-408c7ba95430 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:27.822007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb7190-d786-48e7-a290-408c7ba95430 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:27.822308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb7190-d786-48e7-a290-408c7ba95430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:27.884843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-09eb7190-d786-48e7-a290-408c7ba95430 None None] MFA Rules not processed for user `e9fb56e56c7f49ce959d0bc6c515044c`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:19:27.886608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-09eb7190-d786-48e7-a290-408c7ba95430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:19:27.965302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 748/1504] 10.222.0.64 () {58 vars in 1060 bytes} [Wed Sep 21 16:19:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:19:27.975290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220f457-9c97-48b8-83d7-07e3c6937670 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:19:27.986412 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f220f457-9c97-48b8-83d7-07e3c6937670 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:19:27.989904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014696e-b8ab-4b9f-8e3f-b2cad7df26b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:19:27.990537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014696e-b8ab-4b9f-8e3f-b2cad7df26b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:27.990818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014696e-b8ab-4b9f-8e3f-b2cad7df26b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:27.991093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014696e-b8ab-4b9f-8e3f-b2cad7df26b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:28.000294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014696e-b8ab-4b9f-8e3f-b2cad7df26b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:19:28.001725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014696e-b8ab-4b9f-8e3f-b2cad7df26b5 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:19:28.011443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 757/1505] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:19:27 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:19:28.060224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99e326-da34-4373-83c5-a11cf70ea295 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:28.060669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99e326-da34-4373-83c5-a11cf70ea295 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:28.062040 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99e326-da34-4373-83c5-a11cf70ea295 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:28.068112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 749/1506] 10.222.0.64 () {52 vars in 830 bytes} [Wed Sep 21 16:19:28 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:19:28.072458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014696e-b8ab-4b9f-8e3f-b2cad7df26b5 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:19:28.139098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014696e-b8ab-4b9f-8e3f-b2cad7df26b5 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:19:28.143940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:19:28.144581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:28.144859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:28.145140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] PATH_INFO: `/v3/users/29ad6b0e18f74f0f817c04afbab7a605` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:28.146325 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ad6b0e18f74f0f817c04afbab7a605)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:19:28.147340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:19:28.147340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 21 16:19:28.147340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: Sep 21 16:19:28.148520 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:19:28.220881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ad6b0e18f74f0f817c04afbab7a605'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:19:28.230485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ad6b0e18f74f0f817c04afbab7a605'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:19:28.231125 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ad6b0e18f74f0f817c04afbab7a605'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:19:28.247659 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ad6b0e18f74f0f817c04afbab7a605'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:19:28.269789 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:19:28.272833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ad6b0e18f74f0f817c04afbab7a605'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:19:28.279849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 758/1507] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:19:28 2022] DELETE /identity/v3/users/29ad6b0e18f74f0f817c04afbab7a605 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:19:28.285730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa0323-d54e-4d00-8f21-3b18c6204ac5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:28.285730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa0323-d54e-4d00-8f21-3b18c6204ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:28.285730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa0323-d54e-4d00-8f21-3b18c6204ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:28.308533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3bfa0323-d54e-4d00-8f21-3b18c6204ac5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:19:28.309755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3bfa0323-d54e-4d00-8f21-3b18c6204ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:19:28.320619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:19:28.378513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 750/1508] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:19:28.388352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:19:28.389807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:19:28.478271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1afba-39e7-4c44-9b28-34ab88fe2d40 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:19:28.482591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6247001-2568-40f2-b0a1-20d30657467c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:19:28.483218 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6247001-2568-40f2-b0a1-20d30657467c service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:28.483496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6247001-2568-40f2-b0a1-20d30657467c service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:28.483770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6247001-2568-40f2-b0a1-20d30657467c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:28.493276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6247001-2568-40f2-b0a1-20d30657467c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:19:28.495366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6247001-2568-40f2-b0a1-20d30657467c service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:19:28.505798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 759/1509] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:19:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:19:28.898562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa0323-d54e-4d00-8f21-3b18c6204ac5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:19:28.900684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3bfa0323-d54e-4d00-8f21-3b18c6204ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:19:28.918486 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa0323-d54e-4d00-8f21-3b18c6204ac5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:19:28.925842 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:19:28.926686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:19:28.927067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:19:28.927442 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] PATH_INFO: `/v3/projects/e4fad3f2529048dfa6bca0519627a052` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:19:28.928919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4fad3f2529048dfa6bca0519627a052)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:19:28.930493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:19:28.930493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 21 16:19:28.930493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: Sep 21 16:19:28.933659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:19:29.013794 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4fad3f2529048dfa6bca0519627a052', 'request_id': 'req-4413741e-b158-4463-8a50-b066d411249b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:19:29.025788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4fad3f2529048dfa6bca0519627a052', 'request_id': 'req-4413741e-b158-4463-8a50-b066d411249b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:19:29.033033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4fad3f2529048dfa6bca0519627a052', 'request_id': 'req-4413741e-b158-4463-8a50-b066d411249b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:19:29.057520 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] The token cache is being invalidate because project e4fad3f2529048dfa6bca0519627a052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:19:29.057810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:19:29.058517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4413741e-b158-4463-8a50-b066d411249b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:19:29.060157 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 751/1510] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:19:28 2022] DELETE /identity/v3/projects/e4fad3f2529048dfa6bca0519627a052 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:19:38.500731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:19:39.054188 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:20:55.955853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452eb7ce-48dc-47c5-a8b4-c9674bd88bac None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:20:55.956475 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452eb7ce-48dc-47c5-a8b4-c9674bd88bac None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:20:55.956475 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452eb7ce-48dc-47c5-a8b4-c9674bd88bac None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:20:55.957744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 16:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:20:55.963257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9d407-563d-4f37-8e02-e7f598ccc43c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:20:55.963725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9d407-563d-4f37-8e02-e7f598ccc43c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:20:55.963848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9d407-563d-4f37-8e02-e7f598ccc43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:20:55.983217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-17c9d407-563d-4f37-8e02-e7f598ccc43c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:20:55.984490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-17c9d407-563d-4f37-8e02-e7f598ccc43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:20:56.041880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 16:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:20:56.195497 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452eb7ce-48dc-47c5-a8b4-c9674bd88bac None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:20:56.197155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-452eb7ce-48dc-47c5-a8b4-c9674bd88bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:20:56.296646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452eb7ce-48dc-47c5-a8b4-c9674bd88bac None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:20:56.302501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79663bb2-aac9-42e2-8903-e869c2cce9bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:20:56.303103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79663bb2-aac9-42e2-8903-e869c2cce9bd service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:20:56.303380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79663bb2-aac9-42e2-8903-e869c2cce9bd service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:20:56.303656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79663bb2-aac9-42e2-8903-e869c2cce9bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:20:56.311654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79663bb2-aac9-42e2-8903-e869c2cce9bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:20:56.313221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79663bb2-aac9-42e2-8903-e869c2cce9bd service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:20:56.323116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 761/1513] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:20:57.422891 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871aedda-1564-4c5b-b60d-2c0b113e0792 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:20:57.423455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871aedda-1564-4c5b-b60d-2c0b113e0792 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:20:57.423455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871aedda-1564-4c5b-b60d-2c0b113e0792 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:20:57.424830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 16:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:20:57.431460 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631865e3-a8af-4dcd-9470-1a865c7d9334 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:20:57.431921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631865e3-a8af-4dcd-9470-1a865c7d9334 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:20:57.432047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631865e3-a8af-4dcd-9470-1a865c7d9334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:20:57.451803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-631865e3-a8af-4dcd-9470-1a865c7d9334 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:20:57.453010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-631865e3-a8af-4dcd-9470-1a865c7d9334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:20:57.461271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 16:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:20:57.611438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871aedda-1564-4c5b-b60d-2c0b113e0792 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:20:57.626333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871aedda-1564-4c5b-b60d-2c0b113e0792 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:20:57.632399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a4228-772d-4b1a-a9e1-76c3e679e20b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:20:57.632986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a4228-772d-4b1a-a9e1-76c3e679e20b service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:20:57.633264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a4228-772d-4b1a-a9e1-76c3e679e20b service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:20:57.633537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a4228-772d-4b1a-a9e1-76c3e679e20b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:20:57.641503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a4228-772d-4b1a-a9e1-76c3e679e20b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:20:57.643100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a4228-772d-4b1a-a9e1-76c3e679e20b service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:20:57.652193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 754/1516] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:20:57.856369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d51f17-4701-4a9b-8aac-36d917c04638 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:20:57.856581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d51f17-4701-4a9b-8aac-36d917c04638 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:20:57.856856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d51f17-4701-4a9b-8aac-36d917c04638 None None] PATH_INFO: `/v3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:20:57.858145 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 763/1517] 10.222.0.42 () {54 vars in 925 bytes} [Wed Sep 21 16:20:57 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:20:57.887600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d3bdf-67ea-44a3-89d5-12e07655e968 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:20:57.887886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d3bdf-67ea-44a3-89d5-12e07655e968 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:20:57.888162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438d3bdf-67ea-44a3-89d5-12e07655e968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:20:57.938518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-438d3bdf-67ea-44a3-89d5-12e07655e968 None None] MFA Rules not processed for user `55642294e8ff4f779510f4646e0e30a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:20:57.939713 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-438d3bdf-67ea-44a3-89d5-12e07655e968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:20:58.008793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 755/1518] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Sep 21 16:20:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:21:01.543799 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeac09-14c9-42da-92bc-3310c0bd5221 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:01.544304 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeac09-14c9-42da-92bc-3310c0bd5221 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:01.544436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeac09-14c9-42da-92bc-3310c0bd5221 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:01.545659 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 764/1519] 10.222.0.64 () {54 vars in 961 bytes} [Wed Sep 21 16:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:21:01.551228 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a646-f875-4529-b52c-0266a95a7511 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:01.551714 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a646-f875-4529-b52c-0266a95a7511 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:01.551819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d0a646-f875-4529-b52c-0266a95a7511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:01.572408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-85d0a646-f875-4529-b52c-0266a95a7511 None None] MFA Rules not processed for user `55642294e8ff4f779510f4646e0e30a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:21:01.573651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-85d0a646-f875-4529-b52c-0266a95a7511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:21:01.582344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 756/1520] 10.222.0.64 () {58 vars in 1058 bytes} [Wed Sep 21 16:21:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:21:01.590695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eeac09-14c9-42da-92bc-3310c0bd5221 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:21:01.604884 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eeac09-14c9-42da-92bc-3310c0bd5221 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:21:01.613373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419026c-95bc-4e6e-b7a7-14bded93bbc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:21:01.613978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419026c-95bc-4e6e-b7a7-14bded93bbc9 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:01.614294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419026c-95bc-4e6e-b7a7-14bded93bbc9 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:01.614594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419026c-95bc-4e6e-b7a7-14bded93bbc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:01.623591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419026c-95bc-4e6e-b7a7-14bded93bbc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:21:01.625166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419026c-95bc-4e6e-b7a7-14bded93bbc9 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:21:01.635642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 765/1521] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:21:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:21:09.156655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2c770-df19-422b-8828-0ee0dcdf5906 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:09.157219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2c770-df19-422b-8828-0ee0dcdf5906 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:09.157219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2c770-df19-422b-8828-0ee0dcdf5906 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:09.158548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 16:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:21:09.164704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c7a0f-25d4-44d7-94d0-8278a2dcd831 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:09.165149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c7a0f-25d4-44d7-94d0-8278a2dcd831 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:09.165254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c7a0f-25d4-44d7-94d0-8278a2dcd831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:09.184637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-330c7a0f-25d4-44d7-94d0-8278a2dcd831 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:21:09.185829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-330c7a0f-25d4-44d7-94d0-8278a2dcd831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:21:09.194130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 16:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:21:09.350263 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2c770-df19-422b-8828-0ee0dcdf5906 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:21:09.364934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2c770-df19-422b-8828-0ee0dcdf5906 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:21:09.370448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcd958-83b6-405c-8178-3d0f2d611a1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:21:09.371044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcd958-83b6-405c-8178-3d0f2d611a1c service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:09.371324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcd958-83b6-405c-8178-3d0f2d611a1c service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:09.371597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcd958-83b6-405c-8178-3d0f2d611a1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:09.379682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcd958-83b6-405c-8178-3d0f2d611a1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:21:09.381784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcd958-83b6-405c-8178-3d0f2d611a1c service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:21:09.390959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 758/1524] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:21:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:21:10.462916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea592bc-e211-484a-ad0b-66b08f4d20a2 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:10.463447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea592bc-e211-484a-ad0b-66b08f4d20a2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:10.463447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea592bc-e211-484a-ad0b-66b08f4d20a2 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:10.464779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 21 16:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:21:10.471186 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659799f6-461c-4902-be82-0083d906155c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:10.471584 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659799f6-461c-4902-be82-0083d906155c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:10.471748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659799f6-461c-4902-be82-0083d906155c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:10.492473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-659799f6-461c-4902-be82-0083d906155c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:21:10.493674 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-659799f6-461c-4902-be82-0083d906155c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:21:10.501715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 21 16:21:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:21:10.658556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea592bc-e211-484a-ad0b-66b08f4d20a2 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:21:10.681931 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea592bc-e211-484a-ad0b-66b08f4d20a2 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:21:10.689153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32beb0-456c-4a30-ba76-fdacdd3f93a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:21:10.690489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32beb0-456c-4a30-ba76-fdacdd3f93a1 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:10.690768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32beb0-456c-4a30-ba76-fdacdd3f93a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:10.691036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32beb0-456c-4a30-ba76-fdacdd3f93a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:10.700078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32beb0-456c-4a30-ba76-fdacdd3f93a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:21:10.702040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d32beb0-456c-4a30-ba76-fdacdd3f93a1 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:21:10.713145 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 768/1527] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:21:14.871373 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63aa1db-5282-482d-8c4d-4f7c4b3a00e3 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:14.871913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63aa1db-5282-482d-8c4d-4f7c4b3a00e3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:14.872017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63aa1db-5282-482d-8c4d-4f7c4b3a00e3 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:14.873315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:21:14.878462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299838e-01fb-4f9b-b705-bce07384bd70 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:14.878864 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299838e-01fb-4f9b-b705-bce07384bd70 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:14.879037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299838e-01fb-4f9b-b705-bce07384bd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:14.899243 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1299838e-01fb-4f9b-b705-bce07384bd70 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:21:14.900427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1299838e-01fb-4f9b-b705-bce07384bd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:21:14.908075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:21:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:21:15.195330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63aa1db-5282-482d-8c4d-4f7c4b3a00e3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:21:15.197015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c63aa1db-5282-482d-8c4d-4f7c4b3a00e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:21:15.225439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:21:15.254927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63aa1db-5282-482d-8c4d-4f7c4b3a00e3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:21:15.258445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367f62c-be00-4979-bc2a-095399e8b3d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:21:15.259034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367f62c-be00-4979-bc2a-095399e8b3d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:15.259311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367f62c-be00-4979-bc2a-095399e8b3d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:15.259582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367f62c-be00-4979-bc2a-095399e8b3d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:15.268287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367f62c-be00-4979-bc2a-095399e8b3d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:21:15.269695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367f62c-be00-4979-bc2a-095399e8b3d6 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:21:15.279432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 761/1530] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:21:17.457276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7659c74-f91f-47ab-a370-e697e4237980 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:17.457852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7659c74-f91f-47ab-a370-e697e4237980 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:17.457852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7659c74-f91f-47ab-a370-e697e4237980 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:17.459842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:21:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:21:17.465456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61806a0f-97d1-4e0a-b96b-a5d76d02d51a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:17.465888 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61806a0f-97d1-4e0a-b96b-a5d76d02d51a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:17.465984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61806a0f-97d1-4e0a-b96b-a5d76d02d51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:17.487536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-61806a0f-97d1-4e0a-b96b-a5d76d02d51a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:21:17.488743 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-61806a0f-97d1-4e0a-b96b-a5d76d02d51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:21:17.496526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:21:17.787498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7659c74-f91f-47ab-a370-e697e4237980 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:21:17.798526 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7659c74-f91f-47ab-a370-e697e4237980 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:21:17.802100 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751c9f6-aa10-42f3-8b7e-cdecd4d1c8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:21:17.803240 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751c9f6-aa10-42f3-8b7e-cdecd4d1c8dc service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:17.803524 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751c9f6-aa10-42f3-8b7e-cdecd4d1c8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:17.803800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751c9f6-aa10-42f3-8b7e-cdecd4d1c8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:17.812797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751c9f6-aa10-42f3-8b7e-cdecd4d1c8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:21:17.814275 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751c9f6-aa10-42f3-8b7e-cdecd4d1c8dc service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:21:17.824286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 771/1533] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:21:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:21:21.514689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc083e-a216-4f1f-8726-81f0316d82af None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:21.515222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc083e-a216-4f1f-8726-81f0316d82af None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:21.515222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc083e-a216-4f1f-8726-81f0316d82af None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:21.516563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:21:21.521927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b1965-29c3-4d2e-a243-0eb3694019f2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:21.522382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b1965-29c3-4d2e-a243-0eb3694019f2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:21.522510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b1965-29c3-4d2e-a243-0eb3694019f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:21.530730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=74 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:21:21.544024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8e9b1965-29c3-4d2e-a243-0eb3694019f2 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:21:21.545227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8e9b1965-29c3-4d2e-a243-0eb3694019f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:21:21.553474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:21:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:21:21.858549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc083e-a216-4f1f-8726-81f0316d82af None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:21:21.860423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-89cc083e-a216-4f1f-8726-81f0316d82af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:21:21.918693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc083e-a216-4f1f-8726-81f0316d82af None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:21:21.922275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb58a85-6e9c-4d3d-922a-a7d56cd2109b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:21:21.922886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb58a85-6e9c-4d3d-922a-a7d56cd2109b service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:21.923169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb58a85-6e9c-4d3d-922a-a7d56cd2109b service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:21.923446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb58a85-6e9c-4d3d-922a-a7d56cd2109b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:21.933266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb58a85-6e9c-4d3d-922a-a7d56cd2109b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:21:21.934729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb58a85-6e9c-4d3d-922a-a7d56cd2109b service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:21:21.944563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 764/1536] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:21:24.125380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a733e9f-95a9-4c5f-9ac3-21bdcd5a2528 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:24.125952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a733e9f-95a9-4c5f-9ac3-21bdcd5a2528 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:24.125952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a733e9f-95a9-4c5f-9ac3-21bdcd5a2528 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:24.127286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 21 16:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:21:24.133292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240732f9-24ce-4bc7-a9b7-0c96760ea56f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:24.133796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240732f9-24ce-4bc7-a9b7-0c96760ea56f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:24.133894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240732f9-24ce-4bc7-a9b7-0c96760ea56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:24.154833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-240732f9-24ce-4bc7-a9b7-0c96760ea56f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:21:24.156046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-240732f9-24ce-4bc7-a9b7-0c96760ea56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:21:24.164106 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 21 16:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:21:24.471084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a733e9f-95a9-4c5f-9ac3-21bdcd5a2528 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:21:24.472806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4a733e9f-95a9-4c5f-9ac3-21bdcd5a2528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:21:24.486981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a733e9f-95a9-4c5f-9ac3-21bdcd5a2528 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:21:24.491471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb5fa3-5f93-4f44-b7d1-d9e4a1eb9e3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:21:24.492091 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb5fa3-5f93-4f44-b7d1-d9e4a1eb9e3c service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:21:24.492378 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb5fa3-5f93-4f44-b7d1-d9e4a1eb9e3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:21:24.492651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb5fa3-5f93-4f44-b7d1-d9e4a1eb9e3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:21:24.502331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb5fa3-5f93-4f44-b7d1-d9e4a1eb9e3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:21:24.503766 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb5fa3-5f93-4f44-b7d1-d9e4a1eb9e3c service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:21:24.514508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 774/1539] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:21:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:21:34.149049 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=47 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:21:34.509865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=8 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:48.868328 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a2cfc-6269-4b3c-a22a-a9d9d07a5049 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:48.869462 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a2cfc-6269-4b3c-a22a-a9d9d07a5049 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:48.870061 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a2cfc-6269-4b3c-a22a-a9d9d07a5049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:48.899287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f50a2cfc-6269-4b3c-a22a-a9d9d07a5049 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:48.900932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f50a2cfc-6269-4b3c-a22a-a9d9d07a5049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:48.914989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 766/1540] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:48.925214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a0456-9ef2-4595-9fc5-8229831c98c0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:48.926755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a0456-9ef2-4595-9fc5-8229831c98c0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:48.926881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a0456-9ef2-4595-9fc5-8229831c98c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:48.955770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4b2a0456-9ef2-4595-9fc5-8229831c98c0 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:48.957009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4b2a0456-9ef2-4595-9fc5-8229831c98c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:48.969178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 775/1541] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:48.973743 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a2cfc-6269-4b3c-a22a-a9d9d07a5049 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:48.988041 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a2cfc-6269-4b3c-a22a-a9d9d07a5049 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:48.997618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e508a-b810-4432-bc45-5babe5c9b997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:48.998364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e508a-b810-4432-bc45-5babe5c9b997 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:48.998832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e508a-b810-4432-bc45-5babe5c9b997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:48.999118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e508a-b810-4432-bc45-5babe5c9b997 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:48.999788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e508a-b810-4432-bc45-5babe5c9b997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:22:49.000087 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e508a-b810-4432-bc45-5babe5c9b997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:49.001704 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e508a-b810-4432-bc45-5babe5c9b997 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:49.016298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 767/1542] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:22:49.024729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a0456-9ef2-4595-9fc5-8229831c98c0 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.040356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a0456-9ef2-4595-9fc5-8229831c98c0 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:49.046584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8168ce7-6ed1-4430-b40b-b5a2007471cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:49.047217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8168ce7-6ed1-4430-b40b-b5a2007471cb admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.047494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8168ce7-6ed1-4430-b40b-b5a2007471cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.047763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8168ce7-6ed1-4430-b40b-b5a2007471cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.048407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8168ce7-6ed1-4430-b40b-b5a2007471cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:49.050583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8168ce7-6ed1-4430-b40b-b5a2007471cb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:49.092839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 776/1543] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/projects => generated 367 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:49.097293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e508a-b810-4432-bc45-5babe5c9b997 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.112348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e508a-b810-4432-bc45-5babe5c9b997 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:49.195210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1ccc4-2ffa-4d9e-9bbd-c8fab3150af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:49.195881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1ccc4-2ffa-4d9e-9bbd-c8fab3150af1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.196164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1ccc4-2ffa-4d9e-9bbd-c8fab3150af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.196440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1ccc4-2ffa-4d9e-9bbd-c8fab3150af1 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.197291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1ccc4-2ffa-4d9e-9bbd-c8fab3150af1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:49.199593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1ccc4-2ffa-4d9e-9bbd-c8fab3150af1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:49.228646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 768/1544] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:49.232881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8168ce7-6ed1-4430-b40b-b5a2007471cb admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.252648 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8168ce7-6ed1-4430-b40b-b5a2007471cb admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:49.260360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9855d0-f1f3-4b76-bdc4-87db37706e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:49.260998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9855d0-f1f3-4b76-bdc4-87db37706e36 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.261277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9855d0-f1f3-4b76-bdc4-87db37706e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.261544 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9855d0-f1f3-4b76-bdc4-87db37706e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.262275 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9855d0-f1f3-4b76-bdc4-87db37706e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:49.263929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9855d0-f1f3-4b76-bdc4-87db37706e36 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:49.281375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 777/1545] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:49.285671 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1ccc4-2ffa-4d9e-9bbd-c8fab3150af1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.297025 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1ccc4-2ffa-4d9e-9bbd-c8fab3150af1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:49.305512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cd300-ec39-46a2-aab6-0bb54c0f267f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:49.306249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd300-ec39-46a2-aab6-0bb54c0f267f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.306692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd300-ec39-46a2-aab6-0bb54c0f267f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.306970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd300-ec39-46a2-aab6-0bb54c0f267f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.307619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cd300-ec39-46a2-aab6-0bb54c0f267f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:49.309231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cd300-ec39-46a2-aab6-0bb54c0f267f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:49.321905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 769/1546] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:49.326224 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9855d0-f1f3-4b76-bdc4-87db37706e36 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.343312 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9855d0-f1f3-4b76-bdc4-87db37706e36 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:49.349348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1e7529-6863-4caf-81ca-51fcfd924cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:49.350031 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e7529-6863-4caf-81ca-51fcfd924cfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.350331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e7529-6863-4caf-81ca-51fcfd924cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.350604 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1e7529-6863-4caf-81ca-51fcfd924cfa admin admin] PATH_INFO: `/v3/projects/6d61eb8cae36437eaed5b6498cabb6ee/users/1dd539240b43487380ceabf776fe2c02/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.372739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e7529-6863-4caf-81ca-51fcfd924cfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d61eb8cae36437eaed5b6498cabb6ee, user_id=1dd539240b43487380ceabf776fe2c02, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:49.375954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1e7529-6863-4caf-81ca-51fcfd924cfa admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:49.392204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1b175-7023-47df-a692-a17d92090f65 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.392519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1b175-7023-47df-a692-a17d92090f65 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.392802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde1b175-7023-47df-a692-a17d92090f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.418278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 778/1547] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:22:49 2022] PUT /identity/v3/projects/6d61eb8cae36437eaed5b6498cabb6ee/users/1dd539240b43487380ceabf776fe2c02/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:22:49.428769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fde1b175-7023-47df-a692-a17d92090f65 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.430267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fde1b175-7023-47df-a692-a17d92090f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.475881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47089fb-08fc-4fc4-8e2c-6a8b5dfebf32 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.476084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47089fb-08fc-4fc4-8e2c-6a8b5dfebf32 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.476368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47089fb-08fc-4fc4-8e2c-6a8b5dfebf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.528737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 770/1548] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:22:49.530359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ee449-8f3a-4c9a-846a-974a45601ddd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.530644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ee449-8f3a-4c9a-846a-974a45601ddd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.530914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7ee449-8f3a-4c9a-846a-974a45601ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.535782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c47089fb-08fc-4fc4-8e2c-6a8b5dfebf32 None None] MFA Rules not processed for user `1dd539240b43487380ceabf776fe2c02`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.537418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c47089fb-08fc-4fc4-8e2c-6a8b5dfebf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.565916 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3c7ee449-8f3a-4c9a-846a-974a45601ddd None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.567438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3c7ee449-8f3a-4c9a-846a-974a45601ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.576145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 771/1549] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.577339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66280467-dee2-482d-aef2-9117512fa0e7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.577639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66280467-dee2-482d-aef2-9117512fa0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.577887 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66280467-dee2-482d-aef2-9117512fa0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.608885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-66280467-dee2-482d-aef2-9117512fa0e7 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.610127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-66280467-dee2-482d-aef2-9117512fa0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.618795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 772/1550] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.619663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bd778-90f0-4a57-b55e-6cf0b21d51e6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.619948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bd778-90f0-4a57-b55e-6cf0b21d51e6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.620222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54bd778-90f0-4a57-b55e-6cf0b21d51e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.629210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 779/1551] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:22:49.630442 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e952b-6d45-4220-9edf-e6f0be9e3151 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.630729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e952b-6d45-4220-9edf-e6f0be9e3151 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.631001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e952b-6d45-4220-9edf-e6f0be9e3151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.655436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1f8e952b-6d45-4220-9edf-e6f0be9e3151 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.656645 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1f8e952b-6d45-4220-9edf-e6f0be9e3151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.658515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a54bd778-90f0-4a57-b55e-6cf0b21d51e6 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.659667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a54bd778-90f0-4a57-b55e-6cf0b21d51e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.665023 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 780/1552] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.666274 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9839ddb-c348-4165-bae7-9fc6069b06bf None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.666684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9839ddb-c348-4165-bae7-9fc6069b06bf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.666959 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9839ddb-c348-4165-bae7-9fc6069b06bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.670236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 773/1553] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.671440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06662dde-d448-48f8-ab25-ef4543b8d002 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.671719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06662dde-d448-48f8-ab25-ef4543b8d002 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.671993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06662dde-d448-48f8-ab25-ef4543b8d002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.687940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d9839ddb-c348-4165-bae7-9fc6069b06bf None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.689176 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d9839ddb-c348-4165-bae7-9fc6069b06bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.692438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-06662dde-d448-48f8-ab25-ef4543b8d002 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.693523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-06662dde-d448-48f8-ab25-ef4543b8d002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.697324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 781/1554] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.698780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1076b-6b01-491c-82ef-e535d51e0d2a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.699124 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1076b-6b01-491c-82ef-e535d51e0d2a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.699404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1076b-6b01-491c-82ef-e535d51e0d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.705340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 774/1555] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.705340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36589d-a0c9-4704-ba84-36612cdfbc5f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.705340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36589d-a0c9-4704-ba84-36612cdfbc5f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.705340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36589d-a0c9-4704-ba84-36612cdfbc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.724579 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bf36589d-a0c9-4704-ba84-36612cdfbc5f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.724810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e6b1076b-6b01-491c-82ef-e535d51e0d2a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.725778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bf36589d-a0c9-4704-ba84-36612cdfbc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.725965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e6b1076b-6b01-491c-82ef-e535d51e0d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.733899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 782/1556] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.734197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 775/1557] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.734999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f295a-b88b-40a3-94ce-ee0406b92f31 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.735297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f295a-b88b-40a3-94ce-ee0406b92f31 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.735568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f295a-b88b-40a3-94ce-ee0406b92f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.736094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36589d-a0c9-4704-ba84-36612cdfbc5f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.755227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2d1f295a-b88b-40a3-94ce-ee0406b92f31 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.756433 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2d1f295a-b88b-40a3-94ce-ee0406b92f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.764813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 783/1558] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.765612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f295a-b88b-40a3-94ce-ee0406b92f31 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.770131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:49.776043 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f295a-b88b-40a3-94ce-ee0406b92f31 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:49.781494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90056a6b-0cf1-4d3c-b82b-f186f2977e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:49.782144 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90056a6b-0cf1-4d3c-b82b-f186f2977e42 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.782424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90056a6b-0cf1-4d3c-b82b-f186f2977e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.782696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90056a6b-0cf1-4d3c-b82b-f186f2977e42 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.783349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90056a6b-0cf1-4d3c-b82b-f186f2977e42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:22:49.783653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90056a6b-0cf1-4d3c-b82b-f186f2977e42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:49.784649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-90056a6b-0cf1-4d3c-b82b-f186f2977e42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:22:49.784649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 21 16:22:49.784649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: Sep 21 16:22:49.785533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90056a6b-0cf1-4d3c-b82b-f186f2977e42 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:49.797544 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 784/1559] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:22:49.798997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f023c2-3228-40bc-8ba9-c2eb9f00f2ae None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.799285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f023c2-3228-40bc-8ba9-c2eb9f00f2ae None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.799391 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36589d-a0c9-4704-ba84-36612cdfbc5f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:49.799597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f023c2-3228-40bc-8ba9-c2eb9f00f2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.802928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42202481-b614-4343-9b4d-7048b3e17f85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:49.803541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42202481-b614-4343-9b4d-7048b3e17f85 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.803823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42202481-b614-4343-9b4d-7048b3e17f85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.804094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42202481-b614-4343-9b4d-7048b3e17f85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.812745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42202481-b614-4343-9b4d-7048b3e17f85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:49.814203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42202481-b614-4343-9b4d-7048b3e17f85 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:49.818991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-60f023c2-3228-40bc-8ba9-c2eb9f00f2ae None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.820181 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-60f023c2-3228-40bc-8ba9-c2eb9f00f2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.823454 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 776/1560] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:22:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:22:49.824660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b947a6-3024-412f-a3af-8f3604214dcc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.824945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b947a6-3024-412f-a3af-8f3604214dcc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.825223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b947a6-3024-412f-a3af-8f3604214dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.830325 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 785/1561] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.831542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f023c2-3228-40bc-8ba9-c2eb9f00f2ae None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.843771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f023c2-3228-40bc-8ba9-c2eb9f00f2ae None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:49.846420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-91b947a6-3024-412f-a3af-8f3604214dcc None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.847614 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-91b947a6-3024-412f-a3af-8f3604214dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.848663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a052d-4a5a-4bcd-a241-01e18f9394d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:49.849353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a052d-4a5a-4bcd-a241-01e18f9394d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.849541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a052d-4a5a-4bcd-a241-01e18f9394d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.849968 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a052d-4a5a-4bcd-a241-01e18f9394d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.850569 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a052d-4a5a-4bcd-a241-01e18f9394d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:22:49.850868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a052d-4a5a-4bcd-a241-01e18f9394d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:49.852534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a052d-4a5a-4bcd-a241-01e18f9394d9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:49.856034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 777/1562] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.856979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b947a6-3024-412f-a3af-8f3604214dcc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.865484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 786/1563] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:22:49.866994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38c3a6-4350-4d2e-847f-e413424443a5 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.867286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38c3a6-4350-4d2e-847f-e413424443a5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.867677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b947a6-3024-412f-a3af-8f3604214dcc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:49.867792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38c3a6-4350-4d2e-847f-e413424443a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.876704 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682b492-0401-4482-9986-e51523d06033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:49.877323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682b492-0401-4482-9986-e51523d06033 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.877740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682b492-0401-4482-9986-e51523d06033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.877993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682b492-0401-4482-9986-e51523d06033 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.879419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682b492-0401-4482-9986-e51523d06033 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:22:49.879419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682b492-0401-4482-9986-e51523d06033 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:49.880530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682b492-0401-4482-9986-e51523d06033 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:49.892551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-da38c3a6-4350-4d2e-847f-e413424443a5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.900356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 778/1564] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:22:49.900542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-da38c3a6-4350-4d2e-847f-e413424443a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.900643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29defb1-893c-4882-b9a5-222c03f1f102 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.900643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29defb1-893c-4882-b9a5-222c03f1f102 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.900643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29defb1-893c-4882-b9a5-222c03f1f102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.909480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 787/1565] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.914040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38c3a6-4350-4d2e-847f-e413424443a5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.916637 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f29defb1-893c-4882-b9a5-222c03f1f102 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:49.917828 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f29defb1-893c-4882-b9a5-222c03f1f102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:49.924913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38c3a6-4350-4d2e-847f-e413424443a5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:49.926586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 779/1566] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:49.927321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29defb1-893c-4882-b9a5-222c03f1f102 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.932070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e1748-8ccd-45c9-a2fa-70e9a79688c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:49.932727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e1748-8ccd-45c9-a2fa-70e9a79688c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.933007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e1748-8ccd-45c9-a2fa-70e9a79688c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.933807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e1748-8ccd-45c9-a2fa-70e9a79688c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.934621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e1748-8ccd-45c9-a2fa-70e9a79688c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:49.936572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e1748-8ccd-45c9-a2fa-70e9a79688c7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:49.937131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29defb1-893c-4882-b9a5-222c03f1f102 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:49.942062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c2f97-7c5b-45d0-9e42-e218bb6c068c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:49.942925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c2f97-7c5b-45d0-9e42-e218bb6c068c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:49.943214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c2f97-7c5b-45d0-9e42-e218bb6c068c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:49.943488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c2f97-7c5b-45d0-9e42-e218bb6c068c admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:49.944141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c2f97-7c5b-45d0-9e42-e218bb6c068c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:22:49.944445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c2f97-7c5b-45d0-9e42-e218bb6c068c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:49.946184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c2f97-7c5b-45d0-9e42-e218bb6c068c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:49.958964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:49.960140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 780/1567] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:22:49.961023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c2f97-7c5b-45d0-9e42-e218bb6c068c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.972635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 788/1568] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/projects => generated 389 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:49.974353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e1748-8ccd-45c9-a2fa-70e9a79688c7 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:49.974722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c2f97-7c5b-45d0-9e42-e218bb6c068c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.039428 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29c9d6-1816-4bc2-a5c9-2b805cd3780c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.040036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29c9d6-1816-4bc2-a5c9-2b805cd3780c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.040314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29c9d6-1816-4bc2-a5c9-2b805cd3780c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.040812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29c9d6-1816-4bc2-a5c9-2b805cd3780c admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.041229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29c9d6-1816-4bc2-a5c9-2b805cd3780c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:22:50.041530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29c9d6-1816-4bc2-a5c9-2b805cd3780c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.043547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29c9d6-1816-4bc2-a5c9-2b805cd3780c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.050414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e1748-8ccd-45c9-a2fa-70e9a79688c7 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.056917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a225ce-5ea6-40ea-8b4f-bc94bb064973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.057523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a225ce-5ea6-40ea-8b4f-bc94bb064973 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.057669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 781/1569] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:22:50.057830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a225ce-5ea6-40ea-8b4f-bc94bb064973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.058108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a225ce-5ea6-40ea-8b4f-bc94bb064973 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.058550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29c9d6-1816-4bc2-a5c9-2b805cd3780c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.058835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a225ce-5ea6-40ea-8b4f-bc94bb064973 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.061298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a225ce-5ea6-40ea-8b4f-bc94bb064973 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.070195 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29c9d6-1816-4bc2-a5c9-2b805cd3780c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.075000 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60368d09-1f07-40a5-adc9-41c25ad91a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.075698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60368d09-1f07-40a5-adc9-41c25ad91a00 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.076198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60368d09-1f07-40a5-adc9-41c25ad91a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.077231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60368d09-1f07-40a5-adc9-41c25ad91a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.077942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60368d09-1f07-40a5-adc9-41c25ad91a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.079252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-60368d09-1f07-40a5-adc9-41c25ad91a00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:22:50.079252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 21 16:22:50.079252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: Sep 21 16:22:50.080606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60368d09-1f07-40a5-adc9-41c25ad91a00 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.097410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 789/1570] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:22:49 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.097410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a225ce-5ea6-40ea-8b4f-bc94bb064973 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.116978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 782/1571] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:22:50 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.116978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60368d09-1f07-40a5-adc9-41c25ad91a00 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.175733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a225ce-5ea6-40ea-8b4f-bc94bb064973 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.181751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8338288-021d-4ec1-81e5-0a1590f2e34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.182550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8338288-021d-4ec1-81e5-0a1590f2e34e admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.182929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8338288-021d-4ec1-81e5-0a1590f2e34e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.183263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8338288-021d-4ec1-81e5-0a1590f2e34e admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.183635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60368d09-1f07-40a5-adc9-41c25ad91a00 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.184410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8338288-021d-4ec1-81e5-0a1590f2e34e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:22:50.184773 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8338288-021d-4ec1-81e5-0a1590f2e34e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.187086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8338288-021d-4ec1-81e5-0a1590f2e34e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.188916 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5b142-d0db-4962-ba8c-313aafee8b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.189520 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5b142-d0db-4962-ba8c-313aafee8b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.189800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5b142-d0db-4962-ba8c-313aafee8b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.190101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5b142-d0db-4962-ba8c-313aafee8b7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.190751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5b142-d0db-4962-ba8c-313aafee8b7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:22:50.191054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5b142-d0db-4962-ba8c-313aafee8b7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.192622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5b142-d0db-4962-ba8c-313aafee8b7e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.195435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:50.200549 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 790/1572] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:22:50.201264 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8338288-021d-4ec1-81e5-0a1590f2e34e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.211897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8338288-021d-4ec1-81e5-0a1590f2e34e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.221512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 783/1573] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:22:50.221512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5b142-d0db-4962-ba8c-313aafee8b7e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.221826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c6b5c-cb68-4eee-8017-dc1dbb298ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.221826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c6b5c-cb68-4eee-8017-dc1dbb298ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.221826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c6b5c-cb68-4eee-8017-dc1dbb298ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.221826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c6b5c-cb68-4eee-8017-dc1dbb298ac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.222215 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c6b5c-cb68-4eee-8017-dc1dbb298ac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.223757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c6b5c-cb68-4eee-8017-dc1dbb298ac6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.225756 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5b142-d0db-4962-ba8c-313aafee8b7e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.230394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c15df-4d6b-4a51-8f8a-c204a6888d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.231052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c15df-4d6b-4a51-8f8a-c204a6888d0e admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.231400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c15df-4d6b-4a51-8f8a-c204a6888d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.231736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c15df-4d6b-4a51-8f8a-c204a6888d0e admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.232429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c15df-4d6b-4a51-8f8a-c204a6888d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.236897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c15df-4d6b-4a51-8f8a-c204a6888d0e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.261687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 791/1574] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:22:50 2022] POST /identity/v3/projects => generated 391 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.261687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c6b5c-cb68-4eee-8017-dc1dbb298ac6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.267555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 784/1575] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:22:50 2022] POST /identity/v3/users => generated 295 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.268601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c15df-4d6b-4a51-8f8a-c204a6888d0e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.279921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c15df-4d6b-4a51-8f8a-c204a6888d0e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.353503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:50.364009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c6b5c-cb68-4eee-8017-dc1dbb298ac6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.369221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf62d4-9cf0-4d70-8e86-b5bbe592bc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.369837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ccd71-9887-4943-80df-694f06535b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.370027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf62d4-9cf0-4d70-8e86-b5bbe592bc86 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.370621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ccd71-9887-4943-80df-694f06535b58 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.370751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf62d4-9cf0-4d70-8e86-b5bbe592bc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.371043 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ccd71-9887-4943-80df-694f06535b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.371169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf62d4-9cf0-4d70-8e86-b5bbe592bc86 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.371469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ccd71-9887-4943-80df-694f06535b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.371937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf62d4-9cf0-4d70-8e86-b5bbe592bc86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.372235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ccd71-9887-4943-80df-694f06535b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.374801 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf62d4-9cf0-4d70-8e86-b5bbe592bc86 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.375403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ccd71-9887-4943-80df-694f06535b58 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.405446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 785/1576] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:22:50 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.410807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf62d4-9cf0-4d70-8e86-b5bbe592bc86 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.417219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 792/1577] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:22:50 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.418258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ccd71-9887-4943-80df-694f06535b58 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.427722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf62d4-9cf0-4d70-8e86-b5bbe592bc86 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.504908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ccd71-9887-4943-80df-694f06535b58 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.510719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b2cfd-33d0-4f5c-817b-2e47de3e7e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.511474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b2cfd-33d0-4f5c-817b-2e47de3e7e22 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.511904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b2cfd-33d0-4f5c-817b-2e47de3e7e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.512236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14b2cfd-33d0-4f5c-817b-2e47de3e7e22 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.512983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b2cfd-33d0-4f5c-817b-2e47de3e7e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.513844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eba65-1b08-46c6-a152-873471aae10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.514715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eba65-1b08-46c6-a152-873471aae10d admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.515150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eba65-1b08-46c6-a152-873471aae10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.515292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14b2cfd-33d0-4f5c-817b-2e47de3e7e22 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.515698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9eba65-1b08-46c6-a152-873471aae10d admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.516495 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eba65-1b08-46c6-a152-873471aae10d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.518707 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9eba65-1b08-46c6-a152-873471aae10d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.542508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 786/1578] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:22:50 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.543379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b2cfd-33d0-4f5c-817b-2e47de3e7e22 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.554984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14b2cfd-33d0-4f5c-817b-2e47de3e7e22 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.560311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 793/1579] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:22:50 2022] POST /identity/v3/projects => generated 363 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.560311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eba65-1b08-46c6-a152-873471aae10d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.609434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:50.617661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fa8ad-76ac-4f48-9f19-c1a981131049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.618306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa8ad-76ac-4f48-9f19-c1a981131049 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.618588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa8ad-76ac-4f48-9f19-c1a981131049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.618866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa8ad-76ac-4f48-9f19-c1a981131049 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.619505 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fa8ad-76ac-4f48-9f19-c1a981131049 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.621415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fa8ad-76ac-4f48-9f19-c1a981131049 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.624698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9eba65-1b08-46c6-a152-873471aae10d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.633646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29f5da-2111-460f-b702-528a1c59890e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.634480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29f5da-2111-460f-b702-528a1c59890e admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.634997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29f5da-2111-460f-b702-528a1c59890e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.635488 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29f5da-2111-460f-b702-528a1c59890e admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.636721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29f5da-2111-460f-b702-528a1c59890e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.637852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-2a29f5da-2111-460f-b702-528a1c59890e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:22:50.637852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 21 16:22:50.637852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: Sep 21 16:22:50.639269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29f5da-2111-460f-b702-528a1c59890e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.660203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 787/1580] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:22:50 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.664788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fa8ad-76ac-4f48-9f19-c1a981131049 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.665455 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 794/1581] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:22:50 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.666161 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29f5da-2111-460f-b702-528a1c59890e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.676718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29f5da-2111-460f-b702-528a1c59890e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.730316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:50.731115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fa8ad-76ac-4f48-9f19-c1a981131049 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.736945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c1c67-d8f5-42c8-b2fc-7f42e787fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.737985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c1c67-d8f5-42c8-b2fc-7f42e787fe49 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.738302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c1c67-d8f5-42c8-b2fc-7f42e787fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.738579 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c1c67-d8f5-42c8-b2fc-7f42e787fe49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.739139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5b0c1-6700-4bf3-92ae-24d17e4254ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.739263 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c1c67-d8f5-42c8-b2fc-7f42e787fe49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.740142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5b0c1-6700-4bf3-92ae-24d17e4254ed admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.740440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5b0c1-6700-4bf3-92ae-24d17e4254ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.740884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c1c67-d8f5-42c8-b2fc-7f42e787fe49 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.740998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5b0c1-6700-4bf3-92ae-24d17e4254ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.741574 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5b0c1-6700-4bf3-92ae-24d17e4254ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.743624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5b0c1-6700-4bf3-92ae-24d17e4254ed admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.751907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 788/1582] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:50.752717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c1c67-d8f5-42c8-b2fc-7f42e787fe49 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.754224 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 795/1583] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:50.755333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5b0c1-6700-4bf3-92ae-24d17e4254ed admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.764896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5b0c1-6700-4bf3-92ae-24d17e4254ed admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.766017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c1c67-d8f5-42c8-b2fc-7f42e787fe49 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.769433 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00619a-7d5b-43d8-ba95-6d9959290d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.770033 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00619a-7d5b-43d8-ba95-6d9959290d5c admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.770351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00619a-7d5b-43d8-ba95-6d9959290d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.770622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00619a-7d5b-43d8-ba95-6d9959290d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.771151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7f508-6a5a-40a5-ab69-70df40e685f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.771314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00619a-7d5b-43d8-ba95-6d9959290d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.771751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7f508-6a5a-40a5-ab69-70df40e685f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.772033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7f508-6a5a-40a5-ab69-70df40e685f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.772307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7f508-6a5a-40a5-ab69-70df40e685f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.772804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00619a-7d5b-43d8-ba95-6d9959290d5c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.772944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7f508-6a5a-40a5-ab69-70df40e685f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.774861 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7f508-6a5a-40a5-ab69-70df40e685f4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.785008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 796/1584] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:50.788731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00619a-7d5b-43d8-ba95-6d9959290d5c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.798599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00619a-7d5b-43d8-ba95-6d9959290d5c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.801890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 789/1585] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:22:50 2022] POST /identity/v3/users => generated 289 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.801890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7f508-6a5a-40a5-ab69-70df40e685f4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.803598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d55f27-cd6b-4573-9544-53e808edc482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.804250 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d55f27-cd6b-4573-9544-53e808edc482 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.804504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d55f27-cd6b-4573-9544-53e808edc482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.804779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d55f27-cd6b-4573-9544-53e808edc482 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.805402 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d55f27-cd6b-4573-9544-53e808edc482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.807364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d55f27-cd6b-4573-9544-53e808edc482 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.809854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7f508-6a5a-40a5-ab69-70df40e685f4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.814519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcbafd-df75-4385-ab53-4d24b2c95f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.815126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcbafd-df75-4385-ab53-4d24b2c95f02 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.815595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcbafd-df75-4385-ab53-4d24b2c95f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.815919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcbafd-df75-4385-ab53-4d24b2c95f02 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.816581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcbafd-df75-4385-ab53-4d24b2c95f02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.818532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfcbafd-df75-4385-ab53-4d24b2c95f02 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.830272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 797/1586] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:22:50 2022] POST /identity/v3/users => generated 282 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.831401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d55f27-cd6b-4573-9544-53e808edc482 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.841533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d55f27-cd6b-4573-9544-53e808edc482 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.844320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 790/1587] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:22:50 2022] POST /identity/v3/users => generated 291 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:50.845146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcbafd-df75-4385-ab53-4d24b2c95f02 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.849960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49e610-8975-4db3-a7b4-477f560f60fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.850629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49e610-8975-4db3-a7b4-477f560f60fb admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.850908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49e610-8975-4db3-a7b4-477f560f60fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.851178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49e610-8975-4db3-a7b4-477f560f60fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.851828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49e610-8975-4db3-a7b4-477f560f60fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.853438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49e610-8975-4db3-a7b4-477f560f60fb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.855002 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcbafd-df75-4385-ab53-4d24b2c95f02 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.860705 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b9a91-d883-4377-a183-827e71f6654d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.861304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b9a91-d883-4377-a183-827e71f6654d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.861585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b9a91-d883-4377-a183-827e71f6654d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.861868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b9a91-d883-4377-a183-827e71f6654d admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.862552 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b9a91-d883-4377-a183-827e71f6654d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.864123 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b9a91-d883-4377-a183-827e71f6654d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.864628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 798/1588] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:50.865440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49e610-8975-4db3-a7b4-477f560f60fb admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.875352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49e610-8975-4db3-a7b4-477f560f60fb admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.877945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 791/1589] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:50.880378 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb323dfb-674d-4d89-baac-0a4e9537954f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.881021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb323dfb-674d-4d89-baac-0a4e9537954f admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.881527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb323dfb-674d-4d89-baac-0a4e9537954f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.881829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb323dfb-674d-4d89-baac-0a4e9537954f admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.882011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b9a91-d883-4377-a183-827e71f6654d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.882566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb323dfb-674d-4d89-baac-0a4e9537954f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.884141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb323dfb-674d-4d89-baac-0a4e9537954f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.891589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b9a91-d883-4377-a183-827e71f6654d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.895937 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 799/1590] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:50.899439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87df21-3f30-49ce-8049-f8dbf013419e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.899596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb323dfb-674d-4d89-baac-0a4e9537954f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.899703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87df21-3f30-49ce-8049-f8dbf013419e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.899703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87df21-3f30-49ce-8049-f8dbf013419e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.899703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87df21-3f30-49ce-8049-f8dbf013419e admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.899703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87df21-3f30-49ce-8049-f8dbf013419e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.900063 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87df21-3f30-49ce-8049-f8dbf013419e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.906940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb323dfb-674d-4d89-baac-0a4e9537954f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.914267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 792/1591] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:50.914444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea6916-a487-4ac5-94b2-d309f60135ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.914558 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87df21-3f30-49ce-8049-f8dbf013419e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.914667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea6916-a487-4ac5-94b2-d309f60135ae admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.914667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea6916-a487-4ac5-94b2-d309f60135ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.914667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea6916-a487-4ac5-94b2-d309f60135ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.914667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea6916-a487-4ac5-94b2-d309f60135ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.915162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea6916-a487-4ac5-94b2-d309f60135ae admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.922556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87df21-3f30-49ce-8049-f8dbf013419e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.927660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147330a-a637-47e8-a433-a485d681a2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.929709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147330a-a637-47e8-a433-a485d681a2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.929840 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 800/1592] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:50.929941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147330a-a637-47e8-a433-a485d681a2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.929941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147330a-a637-47e8-a433-a485d681a2f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.930167 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea6916-a487-4ac5-94b2-d309f60135ae admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.930245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147330a-a637-47e8-a433-a485d681a2f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.931300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147330a-a637-47e8-a433-a485d681a2f5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.940435 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea6916-a487-4ac5-94b2-d309f60135ae admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.946825 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed88627-f3dc-4851-8ab5-5348988fcf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.947267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 793/1593] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:50.948104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147330a-a637-47e8-a433-a485d681a2f5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.954377 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed88627-f3dc-4851-8ab5-5348988fcf7b admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.954765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed88627-f3dc-4851-8ab5-5348988fcf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.955077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed88627-f3dc-4851-8ab5-5348988fcf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.955739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed88627-f3dc-4851-8ab5-5348988fcf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.957613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed88627-f3dc-4851-8ab5-5348988fcf7b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.958598 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147330a-a637-47e8-a433-a485d681a2f5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.963577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c176e06-a39d-4cd8-b4d4-f2f4bd1bcecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.964176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c176e06-a39d-4cd8-b4d4-f2f4bd1bcecf admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.964453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c176e06-a39d-4cd8-b4d4-f2f4bd1bcecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.964827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c176e06-a39d-4cd8-b4d4-f2f4bd1bcecf admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.965519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c176e06-a39d-4cd8-b4d4-f2f4bd1bcecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:50.967925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c176e06-a39d-4cd8-b4d4-f2f4bd1bcecf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:50.968896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 801/1594] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:50.969857 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed88627-f3dc-4851-8ab5-5348988fcf7b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.982525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 794/1595] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:50.982525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c176e06-a39d-4cd8-b4d4-f2f4bd1bcecf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:50.983481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed88627-f3dc-4851-8ab5-5348988fcf7b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.988559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae4b8b-9e57-4ad2-96bc-1c91537b009b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.989221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae4b8b-9e57-4ad2-96bc-1c91537b009b admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.989502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae4b8b-9e57-4ad2-96bc-1c91537b009b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.989775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae4b8b-9e57-4ad2-96bc-1c91537b009b admin admin] PATH_INFO: `/v3/projects/0d61f0fe37cf4dc0ba6a2415ddf14518/users/617ba5f9896b457f8182f760ed44aa58/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:50.991225 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c176e06-a39d-4cd8-b4d4-f2f4bd1bcecf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:50.995780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990c7c6-58cc-4b4a-8615-5e50a81cbd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:50.996425 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990c7c6-58cc-4b4a-8615-5e50a81cbd60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:50.996701 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990c7c6-58cc-4b4a-8615-5e50a81cbd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:50.996971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990c7c6-58cc-4b4a-8615-5e50a81cbd60 admin admin] PATH_INFO: `/v3/projects/9c326adfa14149e5ae1d4eb45f4807b8/users/2e5204475ce7477f9cfeeffed457a445/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.004885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae4b8b-9e57-4ad2-96bc-1c91537b009b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d61f0fe37cf4dc0ba6a2415ddf14518, user_id=617ba5f9896b457f8182f760ed44aa58, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.007633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae4b8b-9e57-4ad2-96bc-1c91537b009b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.012107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990c7c6-58cc-4b4a-8615-5e50a81cbd60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c326adfa14149e5ae1d4eb45f4807b8, user_id=2e5204475ce7477f9cfeeffed457a445, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.013242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-4990c7c6-58cc-4b4a-8615-5e50a81cbd60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:22:51.013242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 21 16:22:51.013242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: Sep 21 16:22:51.015313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990c7c6-58cc-4b4a-8615-5e50a81cbd60 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.021394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 802/1596] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:22:50 2022] PUT /identity/v3/projects/0d61f0fe37cf4dc0ba6a2415ddf14518/users/617ba5f9896b457f8182f760ed44aa58/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:22:51.022200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae4b8b-9e57-4ad2-96bc-1c91537b009b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.029072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 795/1597] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:22:50 2022] PUT /identity/v3/projects/9c326adfa14149e5ae1d4eb45f4807b8/users/2e5204475ce7477f9cfeeffed457a445/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:22:51.030492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990c7c6-58cc-4b4a-8615-5e50a81cbd60 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.073260 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:51.091565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae4b8b-9e57-4ad2-96bc-1c91537b009b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.096810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8566fe-1eb2-45a5-bcd0-61f306d428a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.097619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8566fe-1eb2-45a5-bcd0-61f306d428a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.097795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990c7c6-58cc-4b4a-8615-5e50a81cbd60 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.097925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8566fe-1eb2-45a5-bcd0-61f306d428a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.098217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8566fe-1eb2-45a5-bcd0-61f306d428a7 admin admin] PATH_INFO: `/v3/projects/7246fd0179c24bd88ccdfe0926098412/users/68603c7e79184c1b9fd765bedbeb2fa1/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.102779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03597b-c02b-40d3-9f26-62aa51e7c070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.103279 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03597b-c02b-40d3-9f26-62aa51e7c070 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.103559 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03597b-c02b-40d3-9f26-62aa51e7c070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.103830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03597b-c02b-40d3-9f26-62aa51e7c070 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.104474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03597b-c02b-40d3-9f26-62aa51e7c070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.106153 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03597b-c02b-40d3-9f26-62aa51e7c070 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.115594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8566fe-1eb2-45a5-bcd0-61f306d428a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7246fd0179c24bd88ccdfe0926098412, user_id=68603c7e79184c1b9fd765bedbeb2fa1, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.117713 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 796/1598] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:51.118381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8566fe-1eb2-45a5-bcd0-61f306d428a7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.118659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03597b-c02b-40d3-9f26-62aa51e7c070 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.131671 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03597b-c02b-40d3-9f26-62aa51e7c070 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.132923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 803/1599] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:22:51 2022] PUT /identity/v3/projects/7246fd0179c24bd88ccdfe0926098412/users/68603c7e79184c1b9fd765bedbeb2fa1/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:22:51.133970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8566fe-1eb2-45a5-bcd0-61f306d428a7 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.181368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:51.202903 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc62abd-19f6-4acc-ae4c-9014363a0ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.203513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc62abd-19f6-4acc-ae4c-9014363a0ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.203626 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8566fe-1eb2-45a5-bcd0-61f306d428a7 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.203789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc62abd-19f6-4acc-ae4c-9014363a0ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.204058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc62abd-19f6-4acc-ae4c-9014363a0ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.204752 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc62abd-19f6-4acc-ae4c-9014363a0ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.206382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc62abd-19f6-4acc-ae4c-9014363a0ec2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.208232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b67a5e-0439-494c-8fd3-e42c13f7c020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.208845 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b67a5e-0439-494c-8fd3-e42c13f7c020 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.209127 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b67a5e-0439-494c-8fd3-e42c13f7c020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.209401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b67a5e-0439-494c-8fd3-e42c13f7c020 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.210086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b67a5e-0439-494c-8fd3-e42c13f7c020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.211667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b67a5e-0439-494c-8fd3-e42c13f7c020 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.216441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 797/1600] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:51.217963 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc62abd-19f6-4acc-ae4c-9014363a0ec2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.225479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 804/1601] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:51 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:51.226644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99911f08-b0b8-4385-a84e-c732b1ef70f6 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.226935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99911f08-b0b8-4385-a84e-c732b1ef70f6 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.227216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99911f08-b0b8-4385-a84e-c732b1ef70f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.228570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc62abd-19f6-4acc-ae4c-9014363a0ec2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.233472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb1361-dcbc-47b2-95c9-285b501fe11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.234386 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb1361-dcbc-47b2-95c9-285b501fe11c admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.234604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb1361-dcbc-47b2-95c9-285b501fe11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.235009 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb1361-dcbc-47b2-95c9-285b501fe11c admin admin] PATH_INFO: `/v3/projects/b93e202a295d427a85fdaf0209951bd5/users/d2382a6610ae42fa865283ed12dd9982/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.246243 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-99911f08-b0b8-4385-a84e-c732b1ef70f6 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:51.247472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-99911f08-b0b8-4385-a84e-c732b1ef70f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:51.250613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb1361-dcbc-47b2-95c9-285b501fe11c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b93e202a295d427a85fdaf0209951bd5, user_id=d2382a6610ae42fa865283ed12dd9982, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.253272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb1361-dcbc-47b2-95c9-285b501fe11c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.255411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 805/1602] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:51.256389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e58311-cfa7-4034-b7f8-0ab4a87cac2b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.257064 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e58311-cfa7-4034-b7f8-0ab4a87cac2b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.257064 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e58311-cfa7-4034-b7f8-0ab4a87cac2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.270260 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 798/1603] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:22:51 2022] PUT /identity/v3/projects/b93e202a295d427a85fdaf0209951bd5/users/d2382a6610ae42fa865283ed12dd9982/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:22:51.274603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb1361-dcbc-47b2-95c9-285b501fe11c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.277151 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e8e58311-cfa7-4034-b7f8-0ab4a87cac2b None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:51.278373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e8e58311-cfa7-4034-b7f8-0ab4a87cac2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:51.341239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb1361-dcbc-47b2-95c9-285b501fe11c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.342642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 806/1604] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:51.343975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa262fb-ee18-4bbd-b70d-82872bc1a31a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.344269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa262fb-ee18-4bbd-b70d-82872bc1a31a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.344541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa262fb-ee18-4bbd-b70d-82872bc1a31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.349847 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc0eda-4d2a-4eb1-879d-63a957ab571f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.350751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc0eda-4d2a-4eb1-879d-63a957ab571f admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.351244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc0eda-4d2a-4eb1-879d-63a957ab571f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.351718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dc0eda-4d2a-4eb1-879d-63a957ab571f admin admin] PATH_INFO: `/v3/projects/b94ef89f42ce4bc9920e9ac5f1c573e4/users/a4f83757d3484a58b89ec39d46effaab/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.367736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc0eda-4d2a-4eb1-879d-63a957ab571f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b94ef89f42ce4bc9920e9ac5f1c573e4, user_id=a4f83757d3484a58b89ec39d46effaab, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.370459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dc0eda-4d2a-4eb1-879d-63a957ab571f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.372435 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-faa262fb-ee18-4bbd-b70d-82872bc1a31a None None] MFA Rules not processed for user `68603c7e79184c1b9fd765bedbeb2fa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:51.373755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-faa262fb-ee18-4bbd-b70d-82872bc1a31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:51.384316 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 799/1605] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:22:51 2022] PUT /identity/v3/projects/b94ef89f42ce4bc9920e9ac5f1c573e4/users/a4f83757d3484a58b89ec39d46effaab/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:22:51.385027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc0eda-4d2a-4eb1-879d-63a957ab571f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.419131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:51.444471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 807/1606] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:51.445439 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa262fb-ee18-4bbd-b70d-82872bc1a31a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.458038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa262fb-ee18-4bbd-b70d-82872bc1a31a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.458426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dc0eda-4d2a-4eb1-879d-63a957ab571f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.462703 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba520b5-66ad-4900-aa62-08010609ee05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.463027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40787528-de19-410e-8115-4297f2b4347b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.463392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba520b5-66ad-4900-aa62-08010609ee05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.463666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba520b5-66ad-4900-aa62-08010609ee05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.463760 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40787528-de19-410e-8115-4297f2b4347b admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.463946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba520b5-66ad-4900-aa62-08010609ee05 admin admin] PATH_INFO: `/v3/projects/ce7a3e77ddf14b48986752434dda6d33/users/b917d84a650f4355a676e315ea039f0e/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.464058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40787528-de19-410e-8115-4297f2b4347b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.464224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40787528-de19-410e-8115-4297f2b4347b admin admin] PATH_INFO: `/v3/projects/803bc45677c44732a8cf5c18c3910296/users/02812be4572f4918b23c425443f20c2c/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.487662 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40787528-de19-410e-8115-4297f2b4347b admin admin] RBAC: Authorizing `identity:create_grant(project_id=803bc45677c44732a8cf5c18c3910296, user_id=02812be4572f4918b23c425443f20c2c, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.494111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40787528-de19-410e-8115-4297f2b4347b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.496410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba520b5-66ad-4900-aa62-08010609ee05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7a3e77ddf14b48986752434dda6d33, user_id=b917d84a650f4355a676e315ea039f0e, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.500795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba520b5-66ad-4900-aa62-08010609ee05 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.518428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 808/1607] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:22:51 2022] PUT /identity/v3/projects/ce7a3e77ddf14b48986752434dda6d33/users/b917d84a650f4355a676e315ea039f0e/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:22:51.525055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 800/1608] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:22:51 2022] PUT /identity/v3/projects/803bc45677c44732a8cf5c18c3910296/users/02812be4572f4918b23c425443f20c2c/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:22:51.525203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba520b5-66ad-4900-aa62-08010609ee05 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.525314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3da62-8a71-4101-9766-5ebf2a1a89ce None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.525314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3da62-8a71-4101-9766-5ebf2a1a89ce None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.525314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a3da62-8a71-4101-9766-5ebf2a1a89ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.529944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4ba520b5-66ad-4900-aa62-08010609ee05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:51.546988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-63a3da62-8a71-4101-9766-5ebf2a1a89ce None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:51.548221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-63a3da62-8a71-4101-9766-5ebf2a1a89ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:51.575538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:51.602977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba520b5-66ad-4900-aa62-08010609ee05 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.607049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604636c-1a97-4313-b2c8-0a4a73a1a82b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.607701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604636c-1a97-4313-b2c8-0a4a73a1a82b service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.607961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604636c-1a97-4313-b2c8-0a4a73a1a82b service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.608227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604636c-1a97-4313-b2c8-0a4a73a1a82b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.617213 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 801/1609] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:51.618018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3da62-8a71-4101-9766-5ebf2a1a89ce None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.629030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a3da62-8a71-4101-9766-5ebf2a1a89ce None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.633435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9364bf5-958a-4ff8-80e7-a347e824bfc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.634198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9364bf5-958a-4ff8-80e7-a347e824bfc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.635032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9364bf5-958a-4ff8-80e7-a347e824bfc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.635424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9364bf5-958a-4ff8-80e7-a347e824bfc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.647125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9364bf5-958a-4ff8-80e7-a347e824bfc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.648722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9364bf5-958a-4ff8-80e7-a347e824bfc0 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.660242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 802/1610] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:22:51.661656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a5506-dbac-4862-9516-1aeca9358e05 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.662323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a5506-dbac-4862-9516-1aeca9358e05 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.662689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7a5506-dbac-4862-9516-1aeca9358e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.682726 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604636c-1a97-4313-b2c8-0a4a73a1a82b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.683978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ef7a5506-dbac-4862-9516-1aeca9358e05 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:51.684111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604636c-1a97-4313-b2c8-0a4a73a1a82b service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.685312 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ef7a5506-dbac-4862-9516-1aeca9358e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:51.695112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 803/1611] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:51.696861 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029dd49-a786-4783-b21b-b015cae4d482 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.697129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029dd49-a786-4783-b21b-b015cae4d482 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.697473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029dd49-a786-4783-b21b-b015cae4d482 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.698928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 809/1612] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:22:51.700203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbfd4d-6efc-4494-ba2b-6d31e3db50b0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.700610 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbfd4d-6efc-4494-ba2b-6d31e3db50b0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.700987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbfd4d-6efc-4494-ba2b-6d31e3db50b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.707874 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 804/1613] 10.222.0.64 () {54 vars in 965 bytes} [Wed Sep 21 16:22:51 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:22:51.709345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5581e8-f271-4ca3-b1e9-fddb67aed6f0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.709700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5581e8-f271-4ca3-b1e9-fddb67aed6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.710175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5581e8-f271-4ca3-b1e9-fddb67aed6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.728834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5cbbfd4d-6efc-4494-ba2b-6d31e3db50b0 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:51.730197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5cbbfd4d-6efc-4494-ba2b-6d31e3db50b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:51.740268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 810/1614] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:51.743699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbfd4d-6efc-4494-ba2b-6d31e3db50b0 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.754354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbfd4d-6efc-4494-ba2b-6d31e3db50b0 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.754632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cb5581e8-f271-4ca3-b1e9-fddb67aed6f0 None None] MFA Rules not processed for user `02812be4572f4918b23c425443f20c2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:51.755653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cb5581e8-f271-4ca3-b1e9-fddb67aed6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:51.758479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75083066-0309-4fe4-9f12-e7412c860d53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.759190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75083066-0309-4fe4-9f12-e7412c860d53 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.759543 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75083066-0309-4fe4-9f12-e7412c860d53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.759886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75083066-0309-4fe4-9f12-e7412c860d53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.772631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75083066-0309-4fe4-9f12-e7412c860d53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.774624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75083066-0309-4fe4-9f12-e7412c860d53 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.790724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 811/1615] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:22:51.791415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75083066-0309-4fe4-9f12-e7412c860d53 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.802380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75083066-0309-4fe4-9f12-e7412c860d53 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.808778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07a900-9a4d-4ae1-8446-3961287a943a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.809838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07a900-9a4d-4ae1-8446-3961287a943a service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.810392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07a900-9a4d-4ae1-8446-3961287a943a service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.811640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07a900-9a4d-4ae1-8446-3961287a943a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.827084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07a900-9a4d-4ae1-8446-3961287a943a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.828792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07a900-9a4d-4ae1-8446-3961287a943a service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.841859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 812/1616] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:22:51.843587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d4862-6b83-4c0c-bf19-08af442c9bbd None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.843842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d4862-6b83-4c0c-bf19-08af442c9bbd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.844178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d4862-6b83-4c0c-bf19-08af442c9bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.876396 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 805/1617] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:51.877256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5581e8-f271-4ca3-b1e9-fddb67aed6f0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.892416 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5581e8-f271-4ca3-b1e9-fddb67aed6f0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.897208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadaf8d-0528-45b1-9f03-571f305e9dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.898451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadaf8d-0528-45b1-9f03-571f305e9dfb service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.898731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadaf8d-0528-45b1-9f03-571f305e9dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.899111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadaf8d-0528-45b1-9f03-571f305e9dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.903573 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1c3d4862-6b83-4c0c-bf19-08af442c9bbd None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:51.906036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1c3d4862-6b83-4c0c-bf19-08af442c9bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:51.912542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadaf8d-0528-45b1-9f03-571f305e9dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.915111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afadaf8d-0528-45b1-9f03-571f305e9dfb service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.935399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 806/1618] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:22:51.937625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b37d67-51b2-49b5-a339-8d187629a479 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.938076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b37d67-51b2-49b5-a339-8d187629a479 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.938386 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b37d67-51b2-49b5-a339-8d187629a479 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.940848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 807/1619] 10.222.0.64 () {54 vars in 965 bytes} [Wed Sep 21 16:22:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:22:51.941698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b37d67-51b2-49b5-a339-8d187629a479 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:51.960329 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b37d67-51b2-49b5-a339-8d187629a479 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:51.969295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00c048-be7d-4553-babb-bc1bd35217fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:51.970596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00c048-be7d-4553-babb-bc1bd35217fa admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:51.970999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00c048-be7d-4553-babb-bc1bd35217fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:51.971272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00c048-be7d-4553-babb-bc1bd35217fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:51.972615 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00c048-be7d-4553-babb-bc1bd35217fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:51.975833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b00c048-be7d-4553-babb-bc1bd35217fa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:51.984903 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:52.035466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 808/1620] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:22:51 2022] POST /identity/v3/projects => generated 363 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:52.042191 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1845a-5eac-4e68-bfb6-265b4dca697d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.042476 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1845a-5eac-4e68-bfb6-265b4dca697d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.042885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1845a-5eac-4e68-bfb6-265b4dca697d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.049672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 813/1621] 10.222.0.64 () {58 vars in 1062 bytes} [Wed Sep 21 16:22:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:52.050565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d4862-6b83-4c0c-bf19-08af442c9bbd None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:52.067309 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7fe1845a-5eac-4e68-bfb6-265b4dca697d None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:52.068613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7fe1845a-5eac-4e68-bfb6-265b4dca697d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:52.080664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d4862-6b83-4c0c-bf19-08af442c9bbd None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:52.124971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:52.164245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 809/1622] 10.222.0.64 () {58 vars in 1062 bytes} [Wed Sep 21 16:22:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:52.165199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1845a-5eac-4e68-bfb6-265b4dca697d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:52.176751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1845a-5eac-4e68-bfb6-265b4dca697d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:52.182768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163dc16-f37f-4e35-b0e3-44f48557b79a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:52.183033 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eeb8e8-db9e-469c-beef-5bfd4912e207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:52.183419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163dc16-f37f-4e35-b0e3-44f48557b79a service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.183643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163dc16-f37f-4e35-b0e3-44f48557b79a service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.183884 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eeb8e8-db9e-469c-beef-5bfd4912e207 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.184004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163dc16-f37f-4e35-b0e3-44f48557b79a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.184337 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eeb8e8-db9e-469c-beef-5bfd4912e207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.184750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eeb8e8-db9e-469c-beef-5bfd4912e207 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.185658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eeb8e8-db9e-469c-beef-5bfd4912e207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:52.188540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eeb8e8-db9e-469c-beef-5bfd4912e207 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:52.232743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 814/1623] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:22:52 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:22:52.235449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2df4c-ab3e-44f7-86d8-fc4baab54125 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.236274 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2df4c-ab3e-44f7-86d8-fc4baab54125 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.236718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad2df4c-ab3e-44f7-86d8-fc4baab54125 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.242288 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 815/1624] 10.222.0.64 () {54 vars in 912 bytes} [Wed Sep 21 16:22:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:22:52.244557 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2df4c-ab3e-44f7-86d8-fc4baab54125 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:52.268085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad2df4c-ab3e-44f7-86d8-fc4baab54125 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:52.274317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4443c9-81ec-42e5-834f-f9fe8630641a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:52.275385 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4443c9-81ec-42e5-834f-f9fe8630641a admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.275943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4443c9-81ec-42e5-834f-f9fe8630641a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.276411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4443c9-81ec-42e5-834f-f9fe8630641a admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.277102 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4443c9-81ec-42e5-834f-f9fe8630641a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:52.278887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4443c9-81ec-42e5-834f-f9fe8630641a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:52.291467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 816/1625] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:52.293266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170a210-affe-4477-a8d0-0051a45a99e3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.293266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170a210-affe-4477-a8d0-0051a45a99e3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.293266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170a210-affe-4477-a8d0-0051a45a99e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.297563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163dc16-f37f-4e35-b0e3-44f48557b79a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:52.299987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163dc16-f37f-4e35-b0e3-44f48557b79a service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:52.315594 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 810/1626] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:22:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:22:52.320031 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2170a210-affe-4477-a8d0-0051a45a99e3 None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:52.321794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2170a210-affe-4477-a8d0-0051a45a99e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:52.332573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163dc16-f37f-4e35-b0e3-44f48557b79a service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:52.339236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 817/1627] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:22:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:52.353636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170a210-affe-4477-a8d0-0051a45a99e3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:52.365786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163dc16-f37f-4e35-b0e3-44f48557b79a service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:52.372668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3de0e-08f5-4b97-bce9-24f4ee7a3090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:52.372668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3de0e-08f5-4b97-bce9-24f4ee7a3090 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.373642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3de0e-08f5-4b97-bce9-24f4ee7a3090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.373929 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3de0e-08f5-4b97-bce9-24f4ee7a3090 admin admin] PATH_INFO: `/v3/projects/23ff2e5457554972959d09dabd82c22c/users/842a0df169ec44fdb5cfa670fd3c5c68/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.398475 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3de0e-08f5-4b97-bce9-24f4ee7a3090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ff2e5457554972959d09dabd82c22c, user_id=842a0df169ec44fdb5cfa670fd3c5c68, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:52.414815 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff3de0e-08f5-4b97-bce9-24f4ee7a3090 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:52.441718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 811/1628] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:22:52 2022] PUT /identity/v3/projects/23ff2e5457554972959d09dabd82c22c/users/842a0df169ec44fdb5cfa670fd3c5c68/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:22:52.443353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3de0e-08f5-4b97-bce9-24f4ee7a3090 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:52.495016 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170a210-affe-4477-a8d0-0051a45a99e3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:52.547008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:52.569500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff3de0e-08f5-4b97-bce9-24f4ee7a3090 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:52.577116 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7b3c4-987e-47a9-a748-b1b1103c1571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:52.577895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7b3c4-987e-47a9-a748-b1b1103c1571 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.578204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7b3c4-987e-47a9-a748-b1b1103c1571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.578480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7b3c4-987e-47a9-a748-b1b1103c1571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.579135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7b3c4-987e-47a9-a748-b1b1103c1571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:52.581673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7b3c4-987e-47a9-a748-b1b1103c1571 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:52.602831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 812/1629] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:22:52.603999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7b3c4-987e-47a9-a748-b1b1103c1571 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:52.621291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa6e32-d390-488c-8030-c8cbe240dd48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:52.621925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa6e32-d390-488c-8030-c8cbe240dd48 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.622471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa6e32-d390-488c-8030-c8cbe240dd48 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.622471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa6e32-d390-488c-8030-c8cbe240dd48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.629833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7b3c4-987e-47a9-a748-b1b1103c1571 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:52.648864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac6248-62db-4470-9053-dd5ac1fb4c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:52.651567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac6248-62db-4470-9053-dd5ac1fb4c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.651860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac6248-62db-4470-9053-dd5ac1fb4c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.652136 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac6248-62db-4470-9053-dd5ac1fb4c67 admin admin] PATH_INFO: `/v3/projects/23ff2e5457554972959d09dabd82c22c/users/842a0df169ec44fdb5cfa670fd3c5c68/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.657622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac6248-62db-4470-9053-dd5ac1fb4c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ff2e5457554972959d09dabd82c22c, user_id=842a0df169ec44fdb5cfa670fd3c5c68, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:52.661052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cac6248-62db-4470-9053-dd5ac1fb4c67 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:52.698731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 813/1630] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:22:52 2022] PUT /identity/v3/projects/23ff2e5457554972959d09dabd82c22c/users/842a0df169ec44fdb5cfa670fd3c5c68/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:22:52.704656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c533d2-44f8-40c9-a7b4-31636eb9d78a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.705201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c533d2-44f8-40c9-a7b4-31636eb9d78a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.705625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c533d2-44f8-40c9-a7b4-31636eb9d78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.780967 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f4c533d2-44f8-40c9-a7b4-31636eb9d78a None None] MFA Rules not processed for user `842a0df169ec44fdb5cfa670fd3c5c68`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:52.782458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f4c533d2-44f8-40c9-a7b4-31636eb9d78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:52.814555 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa6e32-d390-488c-8030-c8cbe240dd48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:52.816020 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa6e32-d390-488c-8030-c8cbe240dd48 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:52.823123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:52.844339 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 818/1631] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:22:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:22:52.846467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f7b97-2a95-4e82-9d02-f20163163b53 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.847209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f7b97-2a95-4e82-9d02-f20163163b53 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.848893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f7b97-2a95-4e82-9d02-f20163163b53 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.853811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 819/1632] 10.222.0.64 () {54 vars in 912 bytes} [Wed Sep 21 16:22:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:22:52.862959 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26f90-5c5e-4f9b-b0ee-28d216c3dabc None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:52.864301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26f90-5c5e-4f9b-b0ee-28d216c3dabc None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:52.864787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c26f90-5c5e-4f9b-b0ee-28d216c3dabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:52.918453 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b8c26f90-5c5e-4f9b-b0ee-28d216c3dabc None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:52.921457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b8c26f90-5c5e-4f9b-b0ee-28d216c3dabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:52.926129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 814/1633] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:22:52.943578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c533d2-44f8-40c9-a7b4-31636eb9d78a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:53.072868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c533d2-44f8-40c9-a7b4-31636eb9d78a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:53.078430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575e4dd-7873-4d60-a414-4e3b45a066c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:53.079073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575e4dd-7873-4d60-a414-4e3b45a066c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:53.079356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575e4dd-7873-4d60-a414-4e3b45a066c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:53.079634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575e4dd-7873-4d60-a414-4e3b45a066c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:53.097575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575e4dd-7873-4d60-a414-4e3b45a066c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:53.099070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3575e4dd-7873-4d60-a414-4e3b45a066c4 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:53.121180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 815/1634] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:22:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:22:53.209001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 820/1635] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:22:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:22:53.286501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575e4dd-7873-4d60-a414-4e3b45a066c4 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:53.297414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3575e4dd-7873-4d60-a414-4e3b45a066c4 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:53.306720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03c767-1918-41c9-bc37-963bd6e5211b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:53.307342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03c767-1918-41c9-bc37-963bd6e5211b service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:53.307622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03c767-1918-41c9-bc37-963bd6e5211b service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:53.307896 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03c767-1918-41c9-bc37-963bd6e5211b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:53.332726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03c767-1918-41c9-bc37-963bd6e5211b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:53.334209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03c767-1918-41c9-bc37-963bd6e5211b service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:53.358026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 816/1636] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:22:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:22:53.792557 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c26f90-5c5e-4f9b-b0ee-28d216c3dabc None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:53.809534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c26f90-5c5e-4f9b-b0ee-28d216c3dabc None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:53.824874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df117a1-270c-49f5-be6c-01918461f03f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:53.827710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df117a1-270c-49f5-be6c-01918461f03f service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:53.829526 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df117a1-270c-49f5-be6c-01918461f03f service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:53.829823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df117a1-270c-49f5-be6c-01918461f03f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:54.047587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df117a1-270c-49f5-be6c-01918461f03f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:54.049189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df117a1-270c-49f5-be6c-01918461f03f service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:54.073214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 821/1637] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:22:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:22:57.398468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03c767-1918-41c9-bc37-963bd6e5211b service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:57.622932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:22:57.685577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03c767-1918-41c9-bc37-963bd6e5211b service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:57.709413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:57.711977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:57.712277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:57.712538 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] PATH_INFO: `/v3/users/68603c7e79184c1b9fd765bedbeb2fa1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:57.718740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68603c7e79184c1b9fd765bedbeb2fa1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:57.719813 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:22:57.719813 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Sep 21 16:22:57.719813 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: Sep 21 16:22:57.738037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:57.903849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68603c7e79184c1b9fd765bedbeb2fa1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:22:57.922802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68603c7e79184c1b9fd765bedbeb2fa1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:22:57.962832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68603c7e79184c1b9fd765bedbeb2fa1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:22:57.964651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68603c7e79184c1b9fd765bedbeb2fa1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:22:57.988471 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68603c7e79184c1b9fd765bedbeb2fa1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:22:57.997627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 817/1638] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:22:57 2022] DELETE /identity/v3/users/68603c7e79184c1b9fd765bedbeb2fa1 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:22:58.002461 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933a04b-65ac-4406-9cb4-676bff7ef1b0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:58.002957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933a04b-65ac-4406-9cb4-676bff7ef1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:58.003144 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933a04b-65ac-4406-9cb4-676bff7ef1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:58.042781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d933a04b-65ac-4406-9cb4-676bff7ef1b0 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:22:58.044354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d933a04b-65ac-4406-9cb4-676bff7ef1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:58.184235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 822/1639] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:22:58.224224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:22:58.226718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:22:58.443050 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3fbb04-e09c-41bf-9dad-6439bebe12d6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:22:58.452517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f479a09-ecdf-42fe-82dd-806bb14d9f4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:22:58.455065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f479a09-ecdf-42fe-82dd-806bb14d9f4d service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:22:58.456679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f479a09-ecdf-42fe-82dd-806bb14d9f4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:22:58.466303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f479a09-ecdf-42fe-82dd-806bb14d9f4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:22:58.499959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f479a09-ecdf-42fe-82dd-806bb14d9f4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:22:58.503993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f479a09-ecdf-42fe-82dd-806bb14d9f4d service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:22:58.540420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 818/1640] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:22:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:01.276441 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933a04b-65ac-4406-9cb4-676bff7ef1b0 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:01.278158 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d933a04b-65ac-4406-9cb4-676bff7ef1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:01.298749 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933a04b-65ac-4406-9cb4-676bff7ef1b0 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:01.314400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:01.315391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:01.315827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:01.316282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] PATH_INFO: `/v3/projects/7246fd0179c24bd88ccdfe0926098412` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:01.321459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7246fd0179c24bd88ccdfe0926098412)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:01.322885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: WARNING py.warnings [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 21 16:23:01.322885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: warnings.warn(msg) Sep 21 16:23:01.322885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: Sep 21 16:23:01.324492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:01.408137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:01.493623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7246fd0179c24bd88ccdfe0926098412', 'request_id': 'req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:01.514547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7246fd0179c24bd88ccdfe0926098412', 'request_id': 'req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:01.564430 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7246fd0179c24bd88ccdfe0926098412', 'request_id': 'req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:01.613818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] The token cache is being invalidate because project 7246fd0179c24bd88ccdfe0926098412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:01.614454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:01.618656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-81b86ba5-ee88-4985-8ea5-3c2c174d56f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:01.625088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 823/1641] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:01 2022] DELETE /identity/v3/projects/7246fd0179c24bd88ccdfe0926098412 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:01.630138 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43276777-9bc8-41e5-bd80-22ceff2a2adf None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:01.631009 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43276777-9bc8-41e5-bd80-22ceff2a2adf None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:01.631430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43276777-9bc8-41e5-bd80-22ceff2a2adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:01.698837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-43276777-9bc8-41e5-bd80-22ceff2a2adf None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:01.700081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-43276777-9bc8-41e5-bd80-22ceff2a2adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:01.862311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 819/1642] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:01.867850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434b237-02e7-4a73-8b77-d8734c0a3cb4 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:01.868200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434b237-02e7-4a73-8b77-d8734c0a3cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:01.868480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434b237-02e7-4a73-8b77-d8734c0a3cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:01.895813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8434b237-02e7-4a73-8b77-d8734c0a3cb4 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:01.897199 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8434b237-02e7-4a73-8b77-d8734c0a3cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:01.920155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 824/1643] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:01.924138 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43276777-9bc8-41e5-bd80-22ceff2a2adf None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:01.940635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43276777-9bc8-41e5-bd80-22ceff2a2adf None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:01.955695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ebc7b-76ee-49d9-a1cb-def34173efb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:01.957964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ebc7b-76ee-49d9-a1cb-def34173efb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:01.958313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ebc7b-76ee-49d9-a1cb-def34173efb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:01.958633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ebc7b-76ee-49d9-a1cb-def34173efb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:01.961104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ebc7b-76ee-49d9-a1cb-def34173efb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:23:01.961418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ebc7b-76ee-49d9-a1cb-def34173efb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:01.967832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ebc7b-76ee-49d9-a1cb-def34173efb2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:01.986116 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:02.008980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 820/1644] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:23:02.014010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434b237-02e7-4a73-8b77-d8734c0a3cb4 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:02.040863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8434b237-02e7-4a73-8b77-d8734c0a3cb4 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:02.046478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bad56-f5db-4fbc-9862-576ebdd518bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:02.047126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bad56-f5db-4fbc-9862-576ebdd518bb admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:02.047406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bad56-f5db-4fbc-9862-576ebdd518bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:02.047678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3bad56-f5db-4fbc-9862-576ebdd518bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:02.048326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3bad56-f5db-4fbc-9862-576ebdd518bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:02.055686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3bad56-f5db-4fbc-9862-576ebdd518bb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:02.131421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 825/1645] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:02 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:02.136805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ebc7b-76ee-49d9-a1cb-def34173efb2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:02.166646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ebc7b-76ee-49d9-a1cb-def34173efb2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:02.299724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5afd9b-5832-4d35-86dd-a22ed8be1b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:02.300626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5afd9b-5832-4d35-86dd-a22ed8be1b33 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:02.301089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5afd9b-5832-4d35-86dd-a22ed8be1b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:02.301501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5afd9b-5832-4d35-86dd-a22ed8be1b33 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:02.302555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5afd9b-5832-4d35-86dd-a22ed8be1b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:02.305054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5afd9b-5832-4d35-86dd-a22ed8be1b33 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:02.358467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 821/1646] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:02 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:02.361700 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bad56-f5db-4fbc-9862-576ebdd518bb admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:02.387641 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3bad56-f5db-4fbc-9862-576ebdd518bb admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:02.403660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6863d1-a9eb-4e7d-a372-525d13095c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:02.406570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6863d1-a9eb-4e7d-a372-525d13095c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:02.406914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6863d1-a9eb-4e7d-a372-525d13095c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:02.407189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6863d1-a9eb-4e7d-a372-525d13095c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:02.407862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6863d1-a9eb-4e7d-a372-525d13095c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:02.415478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6863d1-a9eb-4e7d-a372-525d13095c7f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:02.467177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 826/1647] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:02.471440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5afd9b-5832-4d35-86dd-a22ed8be1b33 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:02.487946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5afd9b-5832-4d35-86dd-a22ed8be1b33 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:02.515054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cac152-b8cb-4660-aa00-3694ac26de21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:02.520830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cac152-b8cb-4660-aa00-3694ac26de21 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:02.523350 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cac152-b8cb-4660-aa00-3694ac26de21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:02.523638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cac152-b8cb-4660-aa00-3694ac26de21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:02.527137 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cac152-b8cb-4660-aa00-3694ac26de21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:02.529430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cac152-b8cb-4660-aa00-3694ac26de21 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:02.585180 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6863d1-a9eb-4e7d-a372-525d13095c7f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:02.598196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 822/1648] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:02.605509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6863d1-a9eb-4e7d-a372-525d13095c7f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:02.618714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60823e6d-6d24-4d1e-bd81-8724ad0367bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:02.619391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60823e6d-6d24-4d1e-bd81-8724ad0367bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:02.619799 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60823e6d-6d24-4d1e-bd81-8724ad0367bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:02.620082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60823e6d-6d24-4d1e-bd81-8724ad0367bb admin admin] PATH_INFO: `/v3/projects/05c25fd7281e444fbdee942ff9ece3b6/users/6b89e102f3bf468785772b7c195ee68d/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:02.673560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60823e6d-6d24-4d1e-bd81-8724ad0367bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=05c25fd7281e444fbdee942ff9ece3b6, user_id=6b89e102f3bf468785772b7c195ee68d, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:02.680935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60823e6d-6d24-4d1e-bd81-8724ad0367bb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:02.719732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 827/1649] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:02 2022] PUT /identity/v3/projects/05c25fd7281e444fbdee942ff9ece3b6/users/6b89e102f3bf468785772b7c195ee68d/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:02.732116 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fe72b-9222-4488-b06c-e987b73229ba None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:02.732656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fe72b-9222-4488-b06c-e987b73229ba None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:02.733086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fe72b-9222-4488-b06c-e987b73229ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:02.808821 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-161fe72b-9222-4488-b06c-e987b73229ba None None] MFA Rules not processed for user `6b89e102f3bf468785772b7c195ee68d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:02.816409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-161fe72b-9222-4488-b06c-e987b73229ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:03.003021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 823/1650] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:03.004240 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60823e6d-6d24-4d1e-bd81-8724ad0367bb admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:03.236298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60823e6d-6d24-4d1e-bd81-8724ad0367bb admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:03.248207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c33ecc-9576-4bd6-8646-f5b591e9129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:03.248765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c33ecc-9576-4bd6-8646-f5b591e9129f admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:03.249094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c33ecc-9576-4bd6-8646-f5b591e9129f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:03.249430 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c33ecc-9576-4bd6-8646-f5b591e9129f admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:03.253799 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c33ecc-9576-4bd6-8646-f5b591e9129f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:03.261105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c33ecc-9576-4bd6-8646-f5b591e9129f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:03.363262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 828/1651] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:03 2022] POST /identity/v3/projects => generated 381 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:03.367717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fe72b-9222-4488-b06c-e987b73229ba None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:03.391880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fe72b-9222-4488-b06c-e987b73229ba None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:03.621495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ee0c7-0aa1-47fb-b603-baef8a504393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:03.623119 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ee0c7-0aa1-47fb-b603-baef8a504393 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:03.623665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ee0c7-0aa1-47fb-b603-baef8a504393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:03.624068 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1ee0c7-0aa1-47fb-b603-baef8a504393 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:03.624904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ee0c7-0aa1-47fb-b603-baef8a504393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:03.629328 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1ee0c7-0aa1-47fb-b603-baef8a504393 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:03.661155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:03.702144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 824/1652] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:03 2022] POST /identity/v3/users => generated 297 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:03.705821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c33ecc-9576-4bd6-8646-f5b591e9129f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:03.719112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c33ecc-9576-4bd6-8646-f5b591e9129f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:03.729604 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc9a96-55ec-4884-ba70-dbc4c2baa605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:03.730575 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc9a96-55ec-4884-ba70-dbc4c2baa605 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:03.731004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc9a96-55ec-4884-ba70-dbc4c2baa605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:03.731474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc9a96-55ec-4884-ba70-dbc4c2baa605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:03.732375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc9a96-55ec-4884-ba70-dbc4c2baa605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:03.735372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc9a96-55ec-4884-ba70-dbc4c2baa605 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:03.772194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 829/1653] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:03.776512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ee0c7-0aa1-47fb-b603-baef8a504393 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:03.792930 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1ee0c7-0aa1-47fb-b603-baef8a504393 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:03.832492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860be84d-4120-49fb-ad08-edbc3780d640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:03.833322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860be84d-4120-49fb-ad08-edbc3780d640 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:03.833676 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860be84d-4120-49fb-ad08-edbc3780d640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:03.833871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860be84d-4120-49fb-ad08-edbc3780d640 admin admin] PATH_INFO: `/v3/projects/1a133c2bd62543b2b73f49227ad54605/users/ed531602889c437f9701758b3be180be/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:03.894388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860be84d-4120-49fb-ad08-edbc3780d640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a133c2bd62543b2b73f49227ad54605, user_id=ed531602889c437f9701758b3be180be, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:03.897513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860be84d-4120-49fb-ad08-edbc3780d640 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:03.972595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 825/1654] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:03 2022] PUT /identity/v3/projects/1a133c2bd62543b2b73f49227ad54605/users/ed531602889c437f9701758b3be180be/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:03.973861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc9a96-55ec-4884-ba70-dbc4c2baa605 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:03.998083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc9a96-55ec-4884-ba70-dbc4c2baa605 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:04.077545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:04.187097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd1175-0daa-4b65-8da7-fdc26979b9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:04.189174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd1175-0daa-4b65-8da7-fdc26979b9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:04.189469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd1175-0daa-4b65-8da7-fdc26979b9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:04.189775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd1175-0daa-4b65-8da7-fdc26979b9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:04.193063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd1175-0daa-4b65-8da7-fdc26979b9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:04.197047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd1175-0daa-4b65-8da7-fdc26979b9b4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:04.222783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 830/1655] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:04.223439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860be84d-4120-49fb-ad08-edbc3780d640 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:04.239989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860be84d-4120-49fb-ad08-edbc3780d640 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:04.261411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55703ed7-1134-4629-a74d-da130f59f36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:04.271041 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55703ed7-1134-4629-a74d-da130f59f36b admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:04.271614 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55703ed7-1134-4629-a74d-da130f59f36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:04.272043 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55703ed7-1134-4629-a74d-da130f59f36b admin admin] PATH_INFO: `/v3/projects/1a133c2bd62543b2b73f49227ad54605/users/ed531602889c437f9701758b3be180be/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:04.276716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55703ed7-1134-4629-a74d-da130f59f36b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a133c2bd62543b2b73f49227ad54605, user_id=ed531602889c437f9701758b3be180be, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:04.280878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55703ed7-1134-4629-a74d-da130f59f36b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:04.325482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 826/1656] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:04 2022] PUT /identity/v3/projects/1a133c2bd62543b2b73f49227ad54605/users/ed531602889c437f9701758b3be180be/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:04.336155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04b0d1-4da7-4e93-a3b9-3711c153b1b2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:04.336528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04b0d1-4da7-4e93-a3b9-3711c153b1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:04.336874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04b0d1-4da7-4e93-a3b9-3711c153b1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:04.400314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-dd04b0d1-4da7-4e93-a3b9-3711c153b1b2 None None] MFA Rules not processed for user `ed531602889c437f9701758b3be180be`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:04.403954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-dd04b0d1-4da7-4e93-a3b9-3711c153b1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:04.584381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 831/1657] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:04 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:04.598267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55703ed7-1134-4629-a74d-da130f59f36b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:04.600242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-55703ed7-1134-4629-a74d-da130f59f36b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:04.797746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55703ed7-1134-4629-a74d-da130f59f36b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:04.814006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545a545-8f79-4c09-83aa-bfad3d1a3213 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:04.814955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545a545-8f79-4c09-83aa-bfad3d1a3213 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:04.815413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545a545-8f79-4c09-83aa-bfad3d1a3213 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:04.816187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545a545-8f79-4c09-83aa-bfad3d1a3213 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:04.835333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545a545-8f79-4c09-83aa-bfad3d1a3213 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:04.838763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545a545-8f79-4c09-83aa-bfad3d1a3213 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:04.889927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 827/1658] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:04 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:05.034397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04b0d1-4da7-4e93-a3b9-3711c153b1b2 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:05.045636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04b0d1-4da7-4e93-a3b9-3711c153b1b2 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:05.056903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd16395-9753-4adf-8828-0770970ecf5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:05.058244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd16395-9753-4adf-8828-0770970ecf5c service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:05.058630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd16395-9753-4adf-8828-0770970ecf5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:05.059083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd16395-9753-4adf-8828-0770970ecf5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:05.228658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd16395-9753-4adf-8828-0770970ecf5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:05.234193 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd16395-9753-4adf-8828-0770970ecf5c service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:05.266795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 832/1659] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:05.748725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545a545-8f79-4c09-83aa-bfad3d1a3213 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:05.752642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c545a545-8f79-4c09-83aa-bfad3d1a3213 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:05.934646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545a545-8f79-4c09-83aa-bfad3d1a3213 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:05.947762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36942779-8efc-4e1d-a808-c59d9af16f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:05.948744 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36942779-8efc-4e1d-a808-c59d9af16f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:05.949168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36942779-8efc-4e1d-a808-c59d9af16f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:05.949684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36942779-8efc-4e1d-a808-c59d9af16f30 admin admin] PATH_INFO: `/v3/users/1dd539240b43487380ceabf776fe2c02` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:05.951221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36942779-8efc-4e1d-a808-c59d9af16f30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dd539240b43487380ceabf776fe2c02)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:05.953381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36942779-8efc-4e1d-a808-c59d9af16f30 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:05.955932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f7807-4766-4dc8-8df7-716bfe797888 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:05.956412 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f7807-4766-4dc8-8df7-716bfe797888 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:05.956634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f7807-4766-4dc8-8df7-716bfe797888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:06.027214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8c5f7807-4766-4dc8-8df7-716bfe797888 None None] MFA Rules not processed for user `617ba5f9896b457f8182f760ed44aa58`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:06.035181 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8c5f7807-4766-4dc8-8df7-716bfe797888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:06.117469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-36942779-8efc-4e1d-a808-c59d9af16f30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dd539240b43487380ceabf776fe2c02'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:06.137379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-36942779-8efc-4e1d-a808-c59d9af16f30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dd539240b43487380ceabf776fe2c02'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:06.169565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:06.186460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-36942779-8efc-4e1d-a808-c59d9af16f30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dd539240b43487380ceabf776fe2c02'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:06.191864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-36942779-8efc-4e1d-a808-c59d9af16f30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dd539240b43487380ceabf776fe2c02'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:06.198632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:06.225437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-36942779-8efc-4e1d-a808-c59d9af16f30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dd539240b43487380ceabf776fe2c02'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:06.251070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 828/1660] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:05 2022] DELETE /identity/v3/users/1dd539240b43487380ceabf776fe2c02 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:06.259712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3f126-45fb-42fe-bf74-ef2762fe43f7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:06.260008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3f126-45fb-42fe-bf74-ef2762fe43f7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:06.260293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3f126-45fb-42fe-bf74-ef2762fe43f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:06.357934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5ca3f126-45fb-42fe-bf74-ef2762fe43f7 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:06.359287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5ca3f126-45fb-42fe-bf74-ef2762fe43f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:06.362130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 833/1661] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:23:06.382201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f7807-4766-4dc8-8df7-716bfe797888 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:06.386677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8c5f7807-4766-4dc8-8df7-716bfe797888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:06.616327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 829/1662] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:06.681470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f7807-4766-4dc8-8df7-716bfe797888 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:06.722924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3f126-45fb-42fe-bf74-ef2762fe43f7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:06.724539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5ca3f126-45fb-42fe-bf74-ef2762fe43f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:06.826369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea565e34-6f21-4420-ab85-cfa62a721ceb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:06.826708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea565e34-6f21-4420-ab85-cfa62a721ceb service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:06.827042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea565e34-6f21-4420-ab85-cfa62a721ceb service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:06.827290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea565e34-6f21-4420-ab85-cfa62a721ceb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:06.936817 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3f126-45fb-42fe-bf74-ef2762fe43f7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:06.952903 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf7326-ff50-41f3-bf79-1c0fe91d3d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:06.953567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf7326-ff50-41f3-bf79-1c0fe91d3d0f service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:06.954122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf7326-ff50-41f3-bf79-1c0fe91d3d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:06.954416 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf7326-ff50-41f3-bf79-1c0fe91d3d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:06.979252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf7326-ff50-41f3-bf79-1c0fe91d3d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:06.981878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf7326-ff50-41f3-bf79-1c0fe91d3d0f service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:07.012381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea565e34-6f21-4420-ab85-cfa62a721ceb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:07.012935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 830/1663] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:23:07.017268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea565e34-6f21-4420-ab85-cfa62a721ceb service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:07.055788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 834/1664] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:07.287162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf7326-ff50-41f3-bf79-1c0fe91d3d0f service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:07.289243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c0cf7326-ff50-41f3-bf79-1c0fe91d3d0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:07.337892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf7326-ff50-41f3-bf79-1c0fe91d3d0f service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:07.359390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebc0f01-cc1b-4b1c-8fef-53ba6e008008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:07.363037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc0f01-cc1b-4b1c-8fef-53ba6e008008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:07.363805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc0f01-cc1b-4b1c-8fef-53ba6e008008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:07.364782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc0f01-cc1b-4b1c-8fef-53ba6e008008 admin admin] PATH_INFO: `/v3/users/02812be4572f4918b23c425443f20c2c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:07.369372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc0f01-cc1b-4b1c-8fef-53ba6e008008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02812be4572f4918b23c425443f20c2c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:07.379679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebc0f01-cc1b-4b1c-8fef-53ba6e008008 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:07.541722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9ebc0f01-cc1b-4b1c-8fef-53ba6e008008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02812be4572f4918b23c425443f20c2c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:07.580882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9ebc0f01-cc1b-4b1c-8fef-53ba6e008008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02812be4572f4918b23c425443f20c2c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:07.607398 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb64946-2c2b-4220-aa9d-8b86561e2cf1 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:07.611640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb64946-2c2b-4220-aa9d-8b86561e2cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:07.612685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb64946-2c2b-4220-aa9d-8b86561e2cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:07.652256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:07.663922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9ebc0f01-cc1b-4b1c-8fef-53ba6e008008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02812be4572f4918b23c425443f20c2c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:07.667199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9ebc0f01-cc1b-4b1c-8fef-53ba6e008008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02812be4572f4918b23c425443f20c2c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:07.692961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:07.706129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9ebc0f01-cc1b-4b1c-8fef-53ba6e008008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02812be4572f4918b23c425443f20c2c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:07.721859 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 831/1665] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:07 2022] DELETE /identity/v3/users/02812be4572f4918b23c425443f20c2c => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:07.734398 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ebb64946-2c2b-4220-aa9d-8b86561e2cf1 None None] MFA Rules not processed for user `a4f83757d3484a58b89ec39d46effaab`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:07.734811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cc77e-5260-4db8-a885-e49264ff3f04 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:07.735009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ebb64946-2c2b-4220-aa9d-8b86561e2cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:07.735124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cc77e-5260-4db8-a885-e49264ff3f04 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:07.736155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cc77e-5260-4db8-a885-e49264ff3f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:07.807673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-406cc77e-5260-4db8-a885-e49264ff3f04 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:07.808927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-406cc77e-5260-4db8-a885-e49264ff3f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:07.939520 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 835/1666] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:23:07.954495 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e37688-bfe6-4083-a71f-a83a267bba9d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:07.955243 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e37688-bfe6-4083-a71f-a83a267bba9d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:07.955755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e37688-bfe6-4083-a71f-a83a267bba9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:08.041299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-68e37688-bfe6-4083-a71f-a83a267bba9d None None] MFA Rules not processed for user `a4f83757d3484a58b89ec39d46effaab`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:08.044139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-68e37688-bfe6-4083-a71f-a83a267bba9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:08.073157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 836/1667] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:08.081223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 832/1668] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:08.084622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e37688-bfe6-4083-a71f-a83a267bba9d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:08.087902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-68e37688-bfe6-4083-a71f-a83a267bba9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:08.179923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cc77e-5260-4db8-a885-e49264ff3f04 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:08.184111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-406cc77e-5260-4db8-a885-e49264ff3f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:08.363122 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e37688-bfe6-4083-a71f-a83a267bba9d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:08.380197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6c014-6ce6-4e87-8a47-e4eb3277c283 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:08.381563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6c014-6ce6-4e87-8a47-e4eb3277c283 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:08.382220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6c014-6ce6-4e87-8a47-e4eb3277c283 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:08.382928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6c014-6ce6-4e87-8a47-e4eb3277c283 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:08.395608 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6c014-6ce6-4e87-8a47-e4eb3277c283 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:08.398314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6c014-6ce6-4e87-8a47-e4eb3277c283 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:08.412652 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 837/1669] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:08.415306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cc77e-5260-4db8-a885-e49264ff3f04 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:08.419968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1e619-e000-4871-97a1-33ec7b16d5e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:08.420609 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1e619-e000-4871-97a1-33ec7b16d5e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:08.420892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1e619-e000-4871-97a1-33ec7b16d5e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:08.421164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1e619-e000-4871-97a1-33ec7b16d5e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:08.438321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1e619-e000-4871-97a1-33ec7b16d5e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:08.439958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1e619-e000-4871-97a1-33ec7b16d5e5 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:08.455846 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 833/1670] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:09.295192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6c014-6ce6-4e87-8a47-e4eb3277c283 service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:09.296887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4bb6c014-6ce6-4e87-8a47-e4eb3277c283 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:09.335040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6c014-6ce6-4e87-8a47-e4eb3277c283 service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:09.360234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3438b8-bec6-45db-8a66-adce0124a3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:09.361076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3438b8-bec6-45db-8a66-adce0124a3ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:09.361370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3438b8-bec6-45db-8a66-adce0124a3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:09.361648 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3438b8-bec6-45db-8a66-adce0124a3ba admin admin] PATH_INFO: `/v3/projects/6d61eb8cae36437eaed5b6498cabb6ee` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:09.368656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3438b8-bec6-45db-8a66-adce0124a3ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d61eb8cae36437eaed5b6498cabb6ee)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:09.370742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3438b8-bec6-45db-8a66-adce0124a3ba admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:09.427938 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516da8d5-f946-40a9-83c3-515ec554483e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:09.428479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516da8d5-f946-40a9-83c3-515ec554483e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:09.428479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516da8d5-f946-40a9-83c3-515ec554483e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:09.546746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-516da8d5-f946-40a9-83c3-515ec554483e None None] MFA Rules not processed for user `b917d84a650f4355a676e315ea039f0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:09.555868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-516da8d5-f946-40a9-83c3-515ec554483e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:09.586963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fd3438b8-bec6-45db-8a66-adce0124a3ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d61eb8cae36437eaed5b6498cabb6ee', 'request_id': 'req-fd3438b8-bec6-45db-8a66-adce0124a3ba'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:09.624764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fd3438b8-bec6-45db-8a66-adce0124a3ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d61eb8cae36437eaed5b6498cabb6ee', 'request_id': 'req-fd3438b8-bec6-45db-8a66-adce0124a3ba'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:09.658851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fd3438b8-bec6-45db-8a66-adce0124a3ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d61eb8cae36437eaed5b6498cabb6ee', 'request_id': 'req-fd3438b8-bec6-45db-8a66-adce0124a3ba'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:09.692132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:09.699595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fd3438b8-bec6-45db-8a66-adce0124a3ba admin admin] The token cache is being invalidate because project 6d61eb8cae36437eaed5b6498cabb6ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:09.701753 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fd3438b8-bec6-45db-8a66-adce0124a3ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:09.703996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-fd3438b8-bec6-45db-8a66-adce0124a3ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:09.710709 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 838/1671] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:09 2022] DELETE /identity/v3/projects/6d61eb8cae36437eaed5b6498cabb6ee => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:09.717173 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2320a-b206-49d5-ba4d-06e5a47d9904 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:09.717589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2320a-b206-49d5-ba4d-06e5a47d9904 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:09.717935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a2320a-b206-49d5-ba4d-06e5a47d9904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:09.807225 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e6a2320a-b206-49d5-ba4d-06e5a47d9904 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:09.808828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e6a2320a-b206-49d5-ba4d-06e5a47d9904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:09.820912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 834/1672] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:09.829354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516da8d5-f946-40a9-83c3-515ec554483e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:09.829354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-516da8d5-f946-40a9-83c3-515ec554483e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:09.975689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:10.073348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 839/1673] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:10.074432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2320a-b206-49d5-ba4d-06e5a47d9904 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:10.080787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e6a2320a-b206-49d5-ba4d-06e5a47d9904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:10.095649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516da8d5-f946-40a9-83c3-515ec554483e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:10.131467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523cb13-ad92-461f-a8c9-c7669d7419f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:10.132422 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cb13-ad92-461f-a8c9-c7669d7419f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:10.132856 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cb13-ad92-461f-a8c9-c7669d7419f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:10.133275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523cb13-ad92-461f-a8c9-c7669d7419f2 admin admin] PATH_INFO: `/v3/projects/803bc45677c44732a8cf5c18c3910296` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:10.134875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523cb13-ad92-461f-a8c9-c7669d7419f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=803bc45677c44732a8cf5c18c3910296)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:10.137894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523cb13-ad92-461f-a8c9-c7669d7419f2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:10.285696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a2320a-b206-49d5-ba4d-06e5a47d9904 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:10.315449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b523cb13-ad92-461f-a8c9-c7669d7419f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '803bc45677c44732a8cf5c18c3910296', 'request_id': 'req-b523cb13-ad92-461f-a8c9-c7669d7419f2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:10.340045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b523cb13-ad92-461f-a8c9-c7669d7419f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '803bc45677c44732a8cf5c18c3910296', 'request_id': 'req-b523cb13-ad92-461f-a8c9-c7669d7419f2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:10.357714 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b523cb13-ad92-461f-a8c9-c7669d7419f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '803bc45677c44732a8cf5c18c3910296', 'request_id': 'req-b523cb13-ad92-461f-a8c9-c7669d7419f2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:10.394265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b523cb13-ad92-461f-a8c9-c7669d7419f2 admin admin] The token cache is being invalidate because project 803bc45677c44732a8cf5c18c3910296 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:10.400694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b523cb13-ad92-461f-a8c9-c7669d7419f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:10.402851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f7bc3-ed84-45dd-a01a-40bcd1f9696f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:10.404049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f7bc3-ed84-45dd-a01a-40bcd1f9696f service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:10.404330 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f7bc3-ed84-45dd-a01a-40bcd1f9696f service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:10.404595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f7bc3-ed84-45dd-a01a-40bcd1f9696f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:10.412549 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4a0f7bc3-ed84-45dd-a01a-40bcd1f9696f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:10.413685 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b523cb13-ad92-461f-a8c9-c7669d7419f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:10.415937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 835/1674] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:09 2022] DELETE /identity/v3/projects/803bc45677c44732a8cf5c18c3910296 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:10.417516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d513eee-e416-418d-b5b5-3a02e5d37c89 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:10.418037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d513eee-e416-418d-b5b5-3a02e5d37c89 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:10.419426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d513eee-e416-418d-b5b5-3a02e5d37c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:10.455321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5d513eee-e416-418d-b5b5-3a02e5d37c89 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:10.458105 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5d513eee-e416-418d-b5b5-3a02e5d37c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:10.500925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:10.580748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f7bc3-ed84-45dd-a01a-40bcd1f9696f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:10.584124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 836/1675] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:10.586614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f7bc3-ed84-45dd-a01a-40bcd1f9696f service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:10.586978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d513eee-e416-418d-b5b5-3a02e5d37c89 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:10.589922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5d513eee-e416-418d-b5b5-3a02e5d37c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:10.614082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 840/1676] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:10.614620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f7bc3-ed84-45dd-a01a-40bcd1f9696f service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:10.617706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d513eee-e416-418d-b5b5-3a02e5d37c89 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:10.632660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85588b0a-832e-45e4-939c-e84306f77ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:10.633339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85588b0a-832e-45e4-939c-e84306f77ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:10.633619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85588b0a-832e-45e4-939c-e84306f77ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:10.633893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85588b0a-832e-45e4-939c-e84306f77ce6 admin admin] PATH_INFO: `/v3/users/842a0df169ec44fdb5cfa670fd3c5c68` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:10.640197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85588b0a-832e-45e4-939c-e84306f77ce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=842a0df169ec44fdb5cfa670fd3c5c68)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:10.644186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f7bc3-ed84-45dd-a01a-40bcd1f9696f service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:10.646417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85588b0a-832e-45e4-939c-e84306f77ce6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:10.655208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2d10a-c81e-4841-8436-893cdf0bdeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:10.658088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2d10a-c81e-4841-8436-893cdf0bdeba admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:10.659605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2d10a-c81e-4841-8436-893cdf0bdeba admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:10.660129 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2d10a-c81e-4841-8436-893cdf0bdeba admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:10.662299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2d10a-c81e-4841-8436-893cdf0bdeba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:23:10.662824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2d10a-c81e-4841-8436-893cdf0bdeba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:10.664642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2d10a-c81e-4841-8436-893cdf0bdeba admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:10.681215 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 841/1677] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:23:10.685342 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2d10a-c81e-4841-8436-893cdf0bdeba admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:10.713597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2d10a-c81e-4841-8436-893cdf0bdeba admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:10.728160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12661c-20dd-43fa-84c3-8035ef1007e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:10.729037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12661c-20dd-43fa-84c3-8035ef1007e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:10.729463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12661c-20dd-43fa-84c3-8035ef1007e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:10.730635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12661c-20dd-43fa-84c3-8035ef1007e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:10.731992 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12661c-20dd-43fa-84c3-8035ef1007e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:10.737562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b12661c-20dd-43fa-84c3-8035ef1007e7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:10.798151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-85588b0a-832e-45e4-939c-e84306f77ce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842a0df169ec44fdb5cfa670fd3c5c68'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:10.809452 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 842/1678] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:10 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:23:10.810892 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775a9ba-b815-4893-ac89-c883affb88df None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:10.813332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-85588b0a-832e-45e4-939c-e84306f77ce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842a0df169ec44fdb5cfa670fd3c5c68'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:10.813943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775a9ba-b815-4893-ac89-c883affb88df None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:10.814400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775a9ba-b815-4893-ac89-c883affb88df None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:10.815625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:10.818323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 843/1679] 10.222.0.64 () {54 vars in 912 bytes} [Wed Sep 21 16:23:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:23:10.819258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775a9ba-b815-4893-ac89-c883affb88df None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:10.836459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-85588b0a-832e-45e4-939c-e84306f77ce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842a0df169ec44fdb5cfa670fd3c5c68'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:10.837476 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-85588b0a-832e-45e4-939c-e84306f77ce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842a0df169ec44fdb5cfa670fd3c5c68'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:10.850228 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-85588b0a-832e-45e4-939c-e84306f77ce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842a0df169ec44fdb5cfa670fd3c5c68'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:10.872021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 837/1680] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:10 2022] DELETE /identity/v3/users/842a0df169ec44fdb5cfa670fd3c5c68 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:10.876350 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22be4f-18af-4441-931c-b2044f04ac8f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:10.877482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22be4f-18af-4441-931c-b2044f04ac8f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:10.877898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e22be4f-18af-4441-931c-b2044f04ac8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:11.016505 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4e22be4f-18af-4441-931c-b2044f04ac8f None None] MFA Rules not processed for user `e175b8de037d45d48cbddd64e552f6f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:11.018340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4e22be4f-18af-4441-931c-b2044f04ac8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:11.032281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775a9ba-b815-4893-ac89-c883affb88df None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:11.176209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 838/1681] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:23:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:11.292555 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ac079-d492-4971-82e9-748f202ffe33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:11.298366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ac079-d492-4971-82e9-748f202ffe33 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:11.298879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ac079-d492-4971-82e9-748f202ffe33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:11.299369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ac079-d492-4971-82e9-748f202ffe33 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:11.300598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ac079-d492-4971-82e9-748f202ffe33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:11.304530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ac079-d492-4971-82e9-748f202ffe33 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:11.390693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 844/1682] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:10 2022] POST /identity/v3/users => generated 285 bytes in 572 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:11.395521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22be4f-18af-4441-931c-b2044f04ac8f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:11.397576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4e22be4f-18af-4441-931c-b2044f04ac8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:11.419631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ac079-d492-4971-82e9-748f202ffe33 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:11.422034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a75ac079-d492-4971-82e9-748f202ffe33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:11.432345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e22be4f-18af-4441-931c-b2044f04ac8f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:11.444037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ac079-d492-4971-82e9-748f202ffe33 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:11.444975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd3c79-ad94-45a8-94b1-9d102ab23cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:11.445675 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd3c79-ad94-45a8-94b1-9d102ab23cec admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:11.446118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd3c79-ad94-45a8-94b1-9d102ab23cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:11.446508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd3c79-ad94-45a8-94b1-9d102ab23cec admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:11.447275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd3c79-ad94-45a8-94b1-9d102ab23cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:11.449042 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd3c79-ad94-45a8-94b1-9d102ab23cec admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:11.454647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b404f-eaf0-4b0e-9014-fe7176d4faa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:11.456712 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b404f-eaf0-4b0e-9014-fe7176d4faa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:11.457049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b404f-eaf0-4b0e-9014-fe7176d4faa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:11.457335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b404f-eaf0-4b0e-9014-fe7176d4faa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:11.495593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 839/1683] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:11.497485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd3c79-ad94-45a8-94b1-9d102ab23cec admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:11.523195 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b404f-eaf0-4b0e-9014-fe7176d4faa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:11.525821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b404f-eaf0-4b0e-9014-fe7176d4faa7 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:11.530247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd3c79-ad94-45a8-94b1-9d102ab23cec admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:11.545699 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dfa52-4ac9-4ab1-a971-54907df9da81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:11.548420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dfa52-4ac9-4ab1-a971-54907df9da81 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:11.548713 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dfa52-4ac9-4ab1-a971-54907df9da81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:11.550513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dfa52-4ac9-4ab1-a971-54907df9da81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:11.551254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599dfa52-4ac9-4ab1-a971-54907df9da81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:11.558257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599dfa52-4ac9-4ab1-a971-54907df9da81 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:11.569463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 845/1684] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:11.626986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 840/1685] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:11.631694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b404f-eaf0-4b0e-9014-fe7176d4faa7 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:11.650731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b404f-eaf0-4b0e-9014-fe7176d4faa7 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:11.657200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1678adf-f1fb-486e-9728-713c89df6762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:11.658033 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1678adf-f1fb-486e-9728-713c89df6762 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:11.658453 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1678adf-f1fb-486e-9728-713c89df6762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:11.658826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1678adf-f1fb-486e-9728-713c89df6762 admin admin] PATH_INFO: `/v3/projects/9020bde26ac14bba9d04bba7790e01d5/users/1aa0dff88ebe45b09d152a4daee75d1d/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:11.685202 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1678adf-f1fb-486e-9728-713c89df6762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9020bde26ac14bba9d04bba7790e01d5, user_id=1aa0dff88ebe45b09d152a4daee75d1d, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:11.688738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1678adf-f1fb-486e-9728-713c89df6762 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:11.714545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:11.751615 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 846/1686] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:11 2022] PUT /identity/v3/projects/9020bde26ac14bba9d04bba7790e01d5/users/1aa0dff88ebe45b09d152a4daee75d1d/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:11.774652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf614f0-68d8-4adb-9ce2-c8c156aa6db9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:11.777155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf614f0-68d8-4adb-9ce2-c8c156aa6db9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:11.780812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf614f0-68d8-4adb-9ce2-c8c156aa6db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:11.878109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3cf614f0-68d8-4adb-9ce2-c8c156aa6db9 None None] MFA Rules not processed for user `1aa0dff88ebe45b09d152a4daee75d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:11.880698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3cf614f0-68d8-4adb-9ce2-c8c156aa6db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:12.123883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 841/1687] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:12.124735 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1678adf-f1fb-486e-9728-713c89df6762 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:12.299741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1678adf-f1fb-486e-9728-713c89df6762 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:12.306675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd8f8a-0568-4f48-afa0-451ad500c2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:12.307702 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd8f8a-0568-4f48-afa0-451ad500c2ac admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:12.308054 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd8f8a-0568-4f48-afa0-451ad500c2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:12.308381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd8f8a-0568-4f48-afa0-451ad500c2ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:12.309115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd8f8a-0568-4f48-afa0-451ad500c2ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:12.311217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfd8f8a-0568-4f48-afa0-451ad500c2ac admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:12.356487 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 847/1688] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:12 2022] POST /identity/v3/projects => generated 367 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:12.362932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf614f0-68d8-4adb-9ce2-c8c156aa6db9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:12.428287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf614f0-68d8-4adb-9ce2-c8c156aa6db9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:12.542770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:12.599285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f55cf-fd3e-4514-9455-89229fee5a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:12.599285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f55cf-fd3e-4514-9455-89229fee5a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:12.599715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f55cf-fd3e-4514-9455-89229fee5a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:12.599715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f55cf-fd3e-4514-9455-89229fee5a92 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:12.600530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f55cf-fd3e-4514-9455-89229fee5a92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:12.602420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f55cf-fd3e-4514-9455-89229fee5a92 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:12.657624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 842/1689] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:12 2022] POST /identity/v3/users => generated 290 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:12.659003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd8f8a-0568-4f48-afa0-451ad500c2ac admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:12.678833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd8f8a-0568-4f48-afa0-451ad500c2ac admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:12.685111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736119f-e23d-4a87-946a-a6903e33552f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:12.685762 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736119f-e23d-4a87-946a-a6903e33552f admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:12.686126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736119f-e23d-4a87-946a-a6903e33552f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:12.686407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736119f-e23d-4a87-946a-a6903e33552f admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:12.687185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736119f-e23d-4a87-946a-a6903e33552f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:12.689930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736119f-e23d-4a87-946a-a6903e33552f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:12.712684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 848/1690] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:12.719562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f55cf-fd3e-4514-9455-89229fee5a92 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:12.744912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f55cf-fd3e-4514-9455-89229fee5a92 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:12.759559 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02003ee5-5c3e-4573-9a8c-2b8bde418264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:12.760260 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02003ee5-5c3e-4573-9a8c-2b8bde418264 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:12.761014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02003ee5-5c3e-4573-9a8c-2b8bde418264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:12.761014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02003ee5-5c3e-4573-9a8c-2b8bde418264 admin admin] PATH_INFO: `/v3/projects/d8a214c2013b4a1c8e129a0e33a6aa0b/users/d39b4d2528b84a57863e57948dc97912/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:12.814782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02003ee5-5c3e-4573-9a8c-2b8bde418264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a214c2013b4a1c8e129a0e33a6aa0b, user_id=d39b4d2528b84a57863e57948dc97912, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:12.817813 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02003ee5-5c3e-4573-9a8c-2b8bde418264 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:12.883763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 843/1691] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:12 2022] PUT /identity/v3/projects/d8a214c2013b4a1c8e129a0e33a6aa0b/users/d39b4d2528b84a57863e57948dc97912/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:12.889820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736119f-e23d-4a87-946a-a6903e33552f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:12.912137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736119f-e23d-4a87-946a-a6903e33552f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:13.135261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4075454-5216-4bcd-8623-1421690c546f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:13.135980 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4075454-5216-4bcd-8623-1421690c546f admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:13.136167 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4075454-5216-4bcd-8623-1421690c546f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:13.136467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4075454-5216-4bcd-8623-1421690c546f admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:13.137110 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4075454-5216-4bcd-8623-1421690c546f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:13.138847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4075454-5216-4bcd-8623-1421690c546f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:13.183459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 849/1692] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:13.185494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02003ee5-5c3e-4573-9a8c-2b8bde418264 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:13.211363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02003ee5-5c3e-4573-9a8c-2b8bde418264 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:13.256108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63032495-b2ab-4a56-8ebd-b90f66ff88ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:13.262592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63032495-b2ab-4a56-8ebd-b90f66ff88ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:13.263128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63032495-b2ab-4a56-8ebd-b90f66ff88ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:13.263615 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63032495-b2ab-4a56-8ebd-b90f66ff88ef admin admin] PATH_INFO: `/v3/projects/d8a214c2013b4a1c8e129a0e33a6aa0b/users/d39b4d2528b84a57863e57948dc97912/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:13.279798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63032495-b2ab-4a56-8ebd-b90f66ff88ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a214c2013b4a1c8e129a0e33a6aa0b, user_id=d39b4d2528b84a57863e57948dc97912, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:13.296453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63032495-b2ab-4a56-8ebd-b90f66ff88ef admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:13.317366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4075454-5216-4bcd-8623-1421690c546f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:13.320932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a4075454-5216-4bcd-8623-1421690c546f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:13.390189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 844/1693] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:13 2022] PUT /identity/v3/projects/d8a214c2013b4a1c8e129a0e33a6aa0b/users/d39b4d2528b84a57863e57948dc97912/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:13.394113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4075454-5216-4bcd-8623-1421690c546f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:13.399108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35238f11-07d7-4d07-8e93-8f348586ee97 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:13.399542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35238f11-07d7-4d07-8e93-8f348586ee97 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:13.400048 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35238f11-07d7-4d07-8e93-8f348586ee97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:13.615251 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-35238f11-07d7-4d07-8e93-8f348586ee97 None None] MFA Rules not processed for user `d39b4d2528b84a57863e57948dc97912`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:13.620224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-35238f11-07d7-4d07-8e93-8f348586ee97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:13.838997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:13.838997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:13.838997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:13.838997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] PATH_INFO: `/v3/projects/23ff2e5457554972959d09dabd82c22c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:13.849007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ff2e5457554972959d09dabd82c22c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:13.852314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:13.901737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 845/1694] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 504 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:13.903934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7a7f7-9d2a-4447-8f94-3dbf1dfcbf38 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:13.907626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7a7f7-9d2a-4447-8f94-3dbf1dfcbf38 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:13.908841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7a7f7-9d2a-4447-8f94-3dbf1dfcbf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:13.929741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:14.041843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e7e7a7f7-9d2a-4447-8f94-3dbf1dfcbf38 None None] MFA Rules not processed for user `d2382a6610ae42fa865283ed12dd9982`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:14.041843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e7e7a7f7-9d2a-4447-8f94-3dbf1dfcbf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:14.069318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ff2e5457554972959d09dabd82c22c', 'request_id': 'req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:14.091053 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ff2e5457554972959d09dabd82c22c', 'request_id': 'req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:14.102423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ff2e5457554972959d09dabd82c22c', 'request_id': 'req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:14.137942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] The token cache is being invalidate because project 23ff2e5457554972959d09dabd82c22c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:14.138653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:14.140246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:14.143253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 850/1695] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:13 2022] DELETE /identity/v3/projects/23ff2e5457554972959d09dabd82c22c => generated 0 bytes in 827 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:14.144858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:14.149658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:14.194283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 846/1696] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:14.194283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7a7f7-9d2a-4447-8f94-3dbf1dfcbf38 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:14.194283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e7e7a7f7-9d2a-4447-8f94-3dbf1dfcbf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:14.214013 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:14.326075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7a7f7-9d2a-4447-8f94-3dbf1dfcbf38 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:14.331001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f108771f-16b2-4349-8582-631f0dcbee66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:14.331929 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108771f-16b2-4349-8582-631f0dcbee66 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:14.332208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108771f-16b2-4349-8582-631f0dcbee66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:14.332588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f108771f-16b2-4349-8582-631f0dcbee66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:14.340546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f108771f-16b2-4349-8582-631f0dcbee66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:14.425582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fc617-cd21-447a-b228-f9fa0edcf8f9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:14.443283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd4228-06de-4842-bff4-0d39a1a8f9e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:14.443972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd4228-06de-4842-bff4-0d39a1a8f9e7 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:14.444248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd4228-06de-4842-bff4-0d39a1a8f9e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:14.444577 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd4228-06de-4842-bff4-0d39a1a8f9e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:14.448330 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-58dd4228-06de-4842-bff4-0d39a1a8f9e7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:14.476550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd4228-06de-4842-bff4-0d39a1a8f9e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:14.478202 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd4228-06de-4842-bff4-0d39a1a8f9e7 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:14.511829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 851/1697] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:14.514889 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac28cad-3548-4878-a005-53e75ed51713 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:14.518682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac28cad-3548-4878-a005-53e75ed51713 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:14.518970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac28cad-3548-4878-a005-53e75ed51713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:14.528955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108771f-16b2-4349-8582-631f0dcbee66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:14.530378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f108771f-16b2-4349-8582-631f0dcbee66 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:14.563981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 847/1698] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:14.565583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefed79-8a74-44a4-b9d6-b8e41fbc619f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:14.565583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefed79-8a74-44a4-b9d6-b8e41fbc619f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:14.565583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecefed79-8a74-44a4-b9d6-b8e41fbc619f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:14.570272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1ac28cad-3548-4878-a005-53e75ed51713 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:14.571535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1ac28cad-3548-4878-a005-53e75ed51713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:14.640945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ecefed79-8a74-44a4-b9d6-b8e41fbc619f None None] MFA Rules not processed for user `d2382a6610ae42fa865283ed12dd9982`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:14.645346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ecefed79-8a74-44a4-b9d6-b8e41fbc619f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:14.667965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 848/1699] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:14.672201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec274ce0-8ec8-4279-bb76-8d65c310206b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:14.674505 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec274ce0-8ec8-4279-bb76-8d65c310206b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:14.675365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec274ce0-8ec8-4279-bb76-8d65c310206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:14.782689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ec274ce0-8ec8-4279-bb76-8d65c310206b None None] MFA Rules not processed for user `2e5204475ce7477f9cfeeffed457a445`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:14.791640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ec274ce0-8ec8-4279-bb76-8d65c310206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:14.801004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 852/1700] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:14.802098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac28cad-3548-4878-a005-53e75ed51713 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:14.814096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1ac28cad-3548-4878-a005-53e75ed51713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:14.915157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:15.134806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac28cad-3548-4878-a005-53e75ed51713 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:15.146692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 849/1701] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:15.148192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb426c17-bedb-4802-9c2d-5ad22bf7e270 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:15.148578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb426c17-bedb-4802-9c2d-5ad22bf7e270 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:15.148800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb426c17-bedb-4802-9c2d-5ad22bf7e270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:15.177848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a8f87-6c67-46c7-8b4e-38b460ec3222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:15.179001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a8f87-6c67-46c7-8b4e-38b460ec3222 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:15.179001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a8f87-6c67-46c7-8b4e-38b460ec3222 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:15.179226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a8f87-6c67-46c7-8b4e-38b460ec3222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:15.215902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a8f87-6c67-46c7-8b4e-38b460ec3222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:15.217871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a8f87-6c67-46c7-8b4e-38b460ec3222 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:15.228211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-eb426c17-bedb-4802-9c2d-5ad22bf7e270 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:15.229462 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-eb426c17-bedb-4802-9c2d-5ad22bf7e270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:15.242874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 853/1702] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:14 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:15.279298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 850/1703] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:15.282551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a8f87-6c67-46c7-8b4e-38b460ec3222 service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:15.321257 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a8f87-6c67-46c7-8b4e-38b460ec3222 service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:15.352981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30553045-2585-4263-8ac2-ca21c6012c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:15.364917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30553045-2585-4263-8ac2-ca21c6012c14 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:15.366595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30553045-2585-4263-8ac2-ca21c6012c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:15.367365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30553045-2585-4263-8ac2-ca21c6012c14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:15.369009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30553045-2585-4263-8ac2-ca21c6012c14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:23:15.369779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30553045-2585-4263-8ac2-ca21c6012c14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:15.374179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30553045-2585-4263-8ac2-ca21c6012c14 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:15.396955 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 854/1704] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:23:15.407065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb426c17-bedb-4802-9c2d-5ad22bf7e270 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:15.421830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb426c17-bedb-4802-9c2d-5ad22bf7e270 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:15.432316 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fde9d-0422-414a-9055-84f9efaccd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:15.432947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fde9d-0422-414a-9055-84f9efaccd59 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:15.433229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fde9d-0422-414a-9055-84f9efaccd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:15.433503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fde9d-0422-414a-9055-84f9efaccd59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:15.434241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fde9d-0422-414a-9055-84f9efaccd59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:15.436319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fde9d-0422-414a-9055-84f9efaccd59 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:15.522073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 851/1705] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:15 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:15.522756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30553045-2585-4263-8ac2-ca21c6012c14 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:15.546288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fde9d-0422-414a-9055-84f9efaccd59 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:15.552055 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30553045-2585-4263-8ac2-ca21c6012c14 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:15.553425 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-680fde9d-0422-414a-9055-84f9efaccd59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:15.603043 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:15.778932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf15ae-96a2-4a5b-a112-8c495b762515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:15.779824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf15ae-96a2-4a5b-a112-8c495b762515 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:15.780008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf15ae-96a2-4a5b-a112-8c495b762515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:15.780353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf15ae-96a2-4a5b-a112-8c495b762515 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:15.780933 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf15ae-96a2-4a5b-a112-8c495b762515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:15.782950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebf15ae-96a2-4a5b-a112-8c495b762515 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:15.794885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fde9d-0422-414a-9055-84f9efaccd59 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:15.802472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2ed19-776c-4f80-a454-7a4352c145da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:15.811735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2ed19-776c-4f80-a454-7a4352c145da service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:15.812264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2ed19-776c-4f80-a454-7a4352c145da service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:15.812981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2ed19-776c-4f80-a454-7a4352c145da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:15.838630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 855/1706] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:15 2022] POST /identity/v3/users => generated 294 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:15.841547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d2aaa-e966-44a1-a0d4-9ae02299be86 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:15.841745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d2aaa-e966-44a1-a0d4-9ae02299be86 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:15.842021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d2aaa-e966-44a1-a0d4-9ae02299be86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:15.933508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7a7d2aaa-e966-44a1-a0d4-9ae02299be86 None None] MFA Rules not processed for user `617ba5f9896b457f8182f760ed44aa58`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:15.941150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7a7d2aaa-e966-44a1-a0d4-9ae02299be86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:16.036717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2ed19-776c-4f80-a454-7a4352c145da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:16.039462 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2ed19-776c-4f80-a454-7a4352c145da service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:16.092779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 852/1707] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:16.098363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2ed19-776c-4f80-a454-7a4352c145da service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:16.161154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2ed19-776c-4f80-a454-7a4352c145da service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:16.180434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c57936-9314-4cd0-9361-1a34e1ecab7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:16.181676 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c57936-9314-4cd0-9361-1a34e1ecab7a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:16.182185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c57936-9314-4cd0-9361-1a34e1ecab7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:16.189652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c57936-9314-4cd0-9361-1a34e1ecab7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:16.192526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c57936-9314-4cd0-9361-1a34e1ecab7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:16.199280 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c57936-9314-4cd0-9361-1a34e1ecab7a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:16.253199 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 856/1708] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 414 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:16.275081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 853/1709] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:16.281881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d2aaa-e966-44a1-a0d4-9ae02299be86 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:16.384366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d2aaa-e966-44a1-a0d4-9ae02299be86 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:16.404123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb38386-55da-4aec-8a85-4259d73c336b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:16.405621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb38386-55da-4aec-8a85-4259d73c336b admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:16.405998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb38386-55da-4aec-8a85-4259d73c336b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:16.406380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb38386-55da-4aec-8a85-4259d73c336b admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:16.407147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb38386-55da-4aec-8a85-4259d73c336b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:16.409554 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb38386-55da-4aec-8a85-4259d73c336b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:16.451358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 857/1710] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:16.456894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c57936-9314-4cd0-9361-1a34e1ecab7a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:16.476567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c57936-9314-4cd0-9361-1a34e1ecab7a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:16.515731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35a209-2290-4967-bd3e-2198e85dcdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:16.516431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35a209-2290-4967-bd3e-2198e85dcdd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:16.516716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35a209-2290-4967-bd3e-2198e85dcdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:16.516986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35a209-2290-4967-bd3e-2198e85dcdd7 admin admin] PATH_INFO: `/v3/projects/c237ae99818b4d3a8d204cec258c5433/users/cf167fb1986d4ba58fa56c1dc2adf8bb/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:16.559531 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35a209-2290-4967-bd3e-2198e85dcdd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c237ae99818b4d3a8d204cec258c5433, user_id=cf167fb1986d4ba58fa56c1dc2adf8bb, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:16.567566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb38386-55da-4aec-8a85-4259d73c336b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:16.569357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db35a209-2290-4967-bd3e-2198e85dcdd7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:16.570261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2cb38386-55da-4aec-8a85-4259d73c336b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:16.602287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb38386-55da-4aec-8a85-4259d73c336b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:16.603017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 854/1711] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:16 2022] PUT /identity/v3/projects/c237ae99818b4d3a8d204cec258c5433/users/cf167fb1986d4ba58fa56c1dc2adf8bb/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:16.613639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83414f-a246-4038-9c8c-1e1f0ff95d72 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:16.613931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83414f-a246-4038-9c8c-1e1f0ff95d72 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:16.614442 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f83414f-a246-4038-9c8c-1e1f0ff95d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:16.724382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1f83414f-a246-4038-9c8c-1e1f0ff95d72 None None] MFA Rules not processed for user `cf167fb1986d4ba58fa56c1dc2adf8bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:16.725837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1f83414f-a246-4038-9c8c-1e1f0ff95d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:16.732630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:16.805117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d1b84-8742-40a9-aa98-dc8d972b9b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:16.806138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d1b84-8742-40a9-aa98-dc8d972b9b4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:16.806595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d1b84-8742-40a9-aa98-dc8d972b9b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:16.807073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d1b84-8742-40a9-aa98-dc8d972b9b4d admin admin] PATH_INFO: `/v3/users/6b89e102f3bf468785772b7c195ee68d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:16.808647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d1b84-8742-40a9-aa98-dc8d972b9b4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b89e102f3bf468785772b7c195ee68d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:16.811136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d1b84-8742-40a9-aa98-dc8d972b9b4d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:16.928468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-320d1b84-8742-40a9-aa98-dc8d972b9b4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b89e102f3bf468785772b7c195ee68d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:16.945606 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-320d1b84-8742-40a9-aa98-dc8d972b9b4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b89e102f3bf468785772b7c195ee68d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:16.948969 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-320d1b84-8742-40a9-aa98-dc8d972b9b4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b89e102f3bf468785772b7c195ee68d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:16.955132 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:16.979157 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 855/1712] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:23:16.985146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83414f-a246-4038-9c8c-1e1f0ff95d72 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:16.988207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-320d1b84-8742-40a9-aa98-dc8d972b9b4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b89e102f3bf468785772b7c195ee68d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:17.039111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f83414f-a246-4038-9c8c-1e1f0ff95d72 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:17.065123 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdfb01-e448-4547-b741-dbd45b9a4f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:17.066033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdfb01-e448-4547-b741-dbd45b9a4f47 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:17.068130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdfb01-e448-4547-b741-dbd45b9a4f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:17.074102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdfb01-e448-4547-b741-dbd45b9a4f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:17.075522 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdfb01-e448-4547-b741-dbd45b9a4f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:17.079690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-320d1b84-8742-40a9-aa98-dc8d972b9b4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b89e102f3bf468785772b7c195ee68d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:17.089536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdfb01-e448-4547-b741-dbd45b9a4f47 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:17.114006 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 858/1713] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:16 2022] DELETE /identity/v3/users/6b89e102f3bf468785772b7c195ee68d => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:17.116695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d71da-6bf0-4dca-b549-4bf37b5cb704 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:17.119255 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d71da-6bf0-4dca-b549-4bf37b5cb704 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:17.120314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d71da-6bf0-4dca-b549-4bf37b5cb704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:17.193590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-738d71da-6bf0-4dca-b549-4bf37b5cb704 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:17.203132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-738d71da-6bf0-4dca-b549-4bf37b5cb704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:17.212897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 856/1714] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:16 2022] POST /identity/v3/projects => generated 387 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:17.213949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdfb01-e448-4547-b741-dbd45b9a4f47 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:17.222643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-23cdfb01-e448-4547-b741-dbd45b9a4f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:17.471560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 859/1715] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:17.519259 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdfb01-e448-4547-b741-dbd45b9a4f47 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:17.534876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce1348-97ae-449e-b678-30c34f536be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:17.536111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce1348-97ae-449e-b678-30c34f536be8 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:17.537165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce1348-97ae-449e-b678-30c34f536be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:17.537901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce1348-97ae-449e-b678-30c34f536be8 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:17.541589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce1348-97ae-449e-b678-30c34f536be8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:17.546997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce1348-97ae-449e-b678-30c34f536be8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:17.556139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d71da-6bf0-4dca-b549-4bf37b5cb704 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:17.557121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-738d71da-6bf0-4dca-b549-4bf37b5cb704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:17.653895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 857/1716] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:17 2022] POST /identity/v3/users => generated 300 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:17.655562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce1348-97ae-449e-b678-30c34f536be8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:17.703806 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce1348-97ae-449e-b678-30c34f536be8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:17.745732 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f9b6a-5241-45c2-b5f6-06775f6ee775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:17.750513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f9b6a-5241-45c2-b5f6-06775f6ee775 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:17.757714 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f9b6a-5241-45c2-b5f6-06775f6ee775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:17.759848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f9b6a-5241-45c2-b5f6-06775f6ee775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:17.761988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f9b6a-5241-45c2-b5f6-06775f6ee775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:17.770822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750f9b6a-5241-45c2-b5f6-06775f6ee775 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:17.791718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:17.816004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 858/1717] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:23:17.821214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f9b6a-5241-45c2-b5f6-06775f6ee775 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:17.836509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d71da-6bf0-4dca-b549-4bf37b5cb704 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:17.849147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a0a84-d6ec-4fbb-9a59-89e9e2bd3be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:17.852850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a0a84-d6ec-4fbb-9a59-89e9e2bd3be8 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:17.854494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a0a84-d6ec-4fbb-9a59-89e9e2bd3be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:17.856336 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a0a84-d6ec-4fbb-9a59-89e9e2bd3be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:17.859099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f9b6a-5241-45c2-b5f6-06775f6ee775 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:17.885129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1552d61-32ea-425b-b92d-15405edbb448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:17.886027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1552d61-32ea-425b-b92d-15405edbb448 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:17.886755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1552d61-32ea-425b-b92d-15405edbb448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:17.888612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1552d61-32ea-425b-b92d-15405edbb448 admin admin] PATH_INFO: `/v3/projects/80e96a43533f4c299ab83a58a3ae2234/users/da03a393d6ae46228f6cb08eec393450/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:17.897660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a0a84-d6ec-4fbb-9a59-89e9e2bd3be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:17.902206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a0a84-d6ec-4fbb-9a59-89e9e2bd3be8 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:17.953665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1552d61-32ea-425b-b92d-15405edbb448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e96a43533f4c299ab83a58a3ae2234, user_id=da03a393d6ae46228f6cb08eec393450, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:17.957054 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 860/1718] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:17.959628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1552d61-32ea-425b-b92d-15405edbb448 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:17.988926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 859/1719] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:17 2022] PUT /identity/v3/projects/80e96a43533f4c299ab83a58a3ae2234/users/da03a393d6ae46228f6cb08eec393450/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:17.992692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a0a84-d6ec-4fbb-9a59-89e9e2bd3be8 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:18.011466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a0a84-d6ec-4fbb-9a59-89e9e2bd3be8 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:18.164875 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a1511-0342-4ccd-aafa-85266aef06e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:18.165593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a1511-0342-4ccd-aafa-85266aef06e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:18.165870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a1511-0342-4ccd-aafa-85266aef06e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:18.166188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a1511-0342-4ccd-aafa-85266aef06e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:18.166867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a1511-0342-4ccd-aafa-85266aef06e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:18.168873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a1511-0342-4ccd-aafa-85266aef06e3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:18.206410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1552d61-32ea-425b-b92d-15405edbb448 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:18.212779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 861/1720] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:18.252380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1552d61-32ea-425b-b92d-15405edbb448 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:18.306144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532cbb30-513f-4574-a887-4992197817dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:18.327357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532cbb30-513f-4574-a887-4992197817dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:18.338854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532cbb30-513f-4574-a887-4992197817dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:18.339359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532cbb30-513f-4574-a887-4992197817dd admin admin] PATH_INFO: `/v3/projects/80e96a43533f4c299ab83a58a3ae2234/users/da03a393d6ae46228f6cb08eec393450/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:18.357571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532cbb30-513f-4574-a887-4992197817dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=80e96a43533f4c299ab83a58a3ae2234, user_id=da03a393d6ae46228f6cb08eec393450, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:18.368297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532cbb30-513f-4574-a887-4992197817dd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:18.487742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6afad8-7777-4c40-89e1-14ed7a4bb737 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:18.488331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6afad8-7777-4c40-89e1-14ed7a4bb737 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:18.488807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6afad8-7777-4c40-89e1-14ed7a4bb737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:18.521198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 860/1721] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:18 2022] PUT /identity/v3/projects/80e96a43533f4c299ab83a58a3ae2234/users/da03a393d6ae46228f6cb08eec393450/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:18.626780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6c6afad8-7777-4c40-89e1-14ed7a4bb737 None None] MFA Rules not processed for user `da03a393d6ae46228f6cb08eec393450`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:18.632537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6c6afad8-7777-4c40-89e1-14ed7a4bb737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:18.794499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362718c4-1430-4861-a0c9-9db9972d1650 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:18.794936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362718c4-1430-4861-a0c9-9db9972d1650 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:18.795233 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362718c4-1430-4861-a0c9-9db9972d1650 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:18.796688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 861/1722] 10.222.0.64 () {54 vars in 912 bytes} [Wed Sep 21 16:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:23:18.825880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358cab9-4358-4612-aaee-d3a82b194112 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:18.826225 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358cab9-4358-4612-aaee-d3a82b194112 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:18.826514 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358cab9-4358-4612-aaee-d3a82b194112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:18.891136 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4358cab9-4358-4612-aaee-d3a82b194112 None None] MFA Rules not processed for user `e175b8de037d45d48cbddd64e552f6f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:18.897976 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4358cab9-4358-4612-aaee-d3a82b194112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:18.968747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 862/1723] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:19.059703 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6afad8-7777-4c40-89e1-14ed7a4bb737 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:19.062496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6c6afad8-7777-4c40-89e1-14ed7a4bb737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:19.221166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 862/1724] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:23:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:19.223833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358cab9-4358-4612-aaee-d3a82b194112 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:19.247417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4358cab9-4358-4612-aaee-d3a82b194112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:19.356850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6afad8-7777-4c40-89e1-14ed7a4bb737 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:19.369142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43096844-c27a-4186-8a49-fb287a249b75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:19.372771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43096844-c27a-4186-8a49-fb287a249b75 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:19.373043 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43096844-c27a-4186-8a49-fb287a249b75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:19.373360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43096844-c27a-4186-8a49-fb287a249b75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:19.384490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:19.408025 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43096844-c27a-4186-8a49-fb287a249b75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:19.410321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43096844-c27a-4186-8a49-fb287a249b75 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:19.438456 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 863/1725] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:23:19.439234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43096844-c27a-4186-8a49-fb287a249b75 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:19.458987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43096844-c27a-4186-8a49-fb287a249b75 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:19.469401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ead3a-b7e8-4e64-aa82-60d37ab99d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:19.470372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ead3a-b7e8-4e64-aa82-60d37ab99d40 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:19.470844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ead3a-b7e8-4e64-aa82-60d37ab99d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:19.471138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ead3a-b7e8-4e64-aa82-60d37ab99d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:19.493023 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ead3a-b7e8-4e64-aa82-60d37ab99d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:19.494595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ead3a-b7e8-4e64-aa82-60d37ab99d40 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:19.503298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:19.503781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358cab9-4358-4612-aaee-d3a82b194112 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:19.521914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 864/1726] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:19.548082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ead3a-b7e8-4e64-aa82-60d37ab99d40 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:19.552439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c07b9d-8e88-4327-8021-53dd189c2268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:19.553229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c07b9d-8e88-4327-8021-53dd189c2268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:19.553511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c07b9d-8e88-4327-8021-53dd189c2268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:19.553794 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c07b9d-8e88-4327-8021-53dd189c2268 admin admin] PATH_INFO: `/v3/users/1aa0dff88ebe45b09d152a4daee75d1d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:19.559285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c07b9d-8e88-4327-8021-53dd189c2268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aa0dff88ebe45b09d152a4daee75d1d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:19.565285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c07b9d-8e88-4327-8021-53dd189c2268 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:19.578915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ead3a-b7e8-4e64-aa82-60d37ab99d40 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:19.605316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590cb11a-f7c6-4123-a38b-a42dd1ccf338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:19.606759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cb11a-f7c6-4123-a38b-a42dd1ccf338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:19.607047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cb11a-f7c6-4123-a38b-a42dd1ccf338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:19.607324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cb11a-f7c6-4123-a38b-a42dd1ccf338 admin admin] PATH_INFO: `/v3/users/cf167fb1986d4ba58fa56c1dc2adf8bb` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:19.612962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590cb11a-f7c6-4123-a38b-a42dd1ccf338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf167fb1986d4ba58fa56c1dc2adf8bb)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:19.615025 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590cb11a-f7c6-4123-a38b-a42dd1ccf338 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:19.713087 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-02c07b9d-8e88-4327-8021-53dd189c2268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aa0dff88ebe45b09d152a4daee75d1d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:19.742023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-02c07b9d-8e88-4327-8021-53dd189c2268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aa0dff88ebe45b09d152a4daee75d1d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:19.767454 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-02c07b9d-8e88-4327-8021-53dd189c2268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aa0dff88ebe45b09d152a4daee75d1d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:19.768829 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-02c07b9d-8e88-4327-8021-53dd189c2268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aa0dff88ebe45b09d152a4daee75d1d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:19.781829 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-02c07b9d-8e88-4327-8021-53dd189c2268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aa0dff88ebe45b09d152a4daee75d1d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:19.797933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 863/1727] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:19 2022] DELETE /identity/v3/users/1aa0dff88ebe45b09d152a4daee75d1d => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:19.800021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a4def-c59c-4b30-a982-85ae15ff441f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:19.800364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a4def-c59c-4b30-a982-85ae15ff441f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:19.800657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a4def-c59c-4b30-a982-85ae15ff441f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:19.804102 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-590cb11a-f7c6-4123-a38b-a42dd1ccf338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf167fb1986d4ba58fa56c1dc2adf8bb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:19.823553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-590cb11a-f7c6-4123-a38b-a42dd1ccf338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf167fb1986d4ba58fa56c1dc2adf8bb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:19.825536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-590cb11a-f7c6-4123-a38b-a42dd1ccf338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf167fb1986d4ba58fa56c1dc2adf8bb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:19.849217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-590cb11a-f7c6-4123-a38b-a42dd1ccf338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf167fb1986d4ba58fa56c1dc2adf8bb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:19.887627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-590cb11a-f7c6-4123-a38b-a42dd1ccf338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf167fb1986d4ba58fa56c1dc2adf8bb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:19.903467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 865/1728] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:19 2022] DELETE /identity/v3/users/cf167fb1986d4ba58fa56c1dc2adf8bb => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:19.912235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376df31-4d36-4bdf-bb47-6a2a4c964990 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:19.914323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376df31-4d36-4bdf-bb47-6a2a4c964990 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:19.914957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376df31-4d36-4bdf-bb47-6a2a4c964990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:19.971741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3b1a4def-c59c-4b30-a982-85ae15ff441f None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:19.973868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3b1a4def-c59c-4b30-a982-85ae15ff441f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:19.996674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8376df31-4d36-4bdf-bb47-6a2a4c964990 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:20.001762 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8376df31-4d36-4bdf-bb47-6a2a4c964990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:20.150493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 864/1729] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:23:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:20.154374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495261a-2a30-4241-b730-d7994e64debb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:20.156641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495261a-2a30-4241-b730-d7994e64debb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:20.157340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495261a-2a30-4241-b730-d7994e64debb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:20.196971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 866/1730] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:20.236256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376df31-4d36-4bdf-bb47-6a2a4c964990 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:20.237785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8376df31-4d36-4bdf-bb47-6a2a4c964990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:20.241739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5495261a-2a30-4241-b730-d7994e64debb None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:20.248138 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5495261a-2a30-4241-b730-d7994e64debb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:20.292568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 865/1731] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:20.295098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b737512-a2b6-4d41-b00a-ae8927cc96fc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:20.302235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b737512-a2b6-4d41-b00a-ae8927cc96fc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:20.303753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b737512-a2b6-4d41-b00a-ae8927cc96fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:20.341105 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8b737512-a2b6-4d41-b00a-ae8927cc96fc None None] MFA Rules not processed for user `b917d84a650f4355a676e315ea039f0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:20.343567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8b737512-a2b6-4d41-b00a-ae8927cc96fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:20.352057 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:20.433058 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376df31-4d36-4bdf-bb47-6a2a4c964990 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:20.444975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396b463-3a9f-45f1-ab5c-b039f078b8b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:20.446455 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396b463-3a9f-45f1-ab5c-b039f078b8b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:20.446963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396b463-3a9f-45f1-ab5c-b039f078b8b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:20.447244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6396b463-3a9f-45f1-ab5c-b039f078b8b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:20.468201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396b463-3a9f-45f1-ab5c-b039f078b8b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:20.470820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6396b463-3a9f-45f1-ab5c-b039f078b8b9 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:20.483191 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 866/1732] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:20.484883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153bda1-0826-4afd-a0a9-9b371f7ef7d7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:20.485436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153bda1-0826-4afd-a0a9-9b371f7ef7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:20.485828 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a153bda1-0826-4afd-a0a9-9b371f7ef7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:20.506957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 867/1733] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:20.510373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396b463-3a9f-45f1-ab5c-b039f078b8b9 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:20.535110 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6396b463-3a9f-45f1-ab5c-b039f078b8b9 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:20.550116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd12332-1d02-4ed7-a986-f5fbda0881e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:20.551098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd12332-1d02-4ed7-a986-f5fbda0881e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:20.552761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd12332-1d02-4ed7-a986-f5fbda0881e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:20.553289 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd12332-1d02-4ed7-a986-f5fbda0881e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:20.582645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a153bda1-0826-4afd-a0a9-9b371f7ef7d7 None None] MFA Rules not processed for user `d2382a6610ae42fa865283ed12dd9982`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:20.586897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a153bda1-0826-4afd-a0a9-9b371f7ef7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:20.590959 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd12332-1d02-4ed7-a986-f5fbda0881e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:20.592462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd12332-1d02-4ed7-a986-f5fbda0881e6 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:20.612303 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:20.641115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 868/1734] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:20.642451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd12332-1d02-4ed7-a986-f5fbda0881e6 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:20.650877 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2dd12332-1d02-4ed7-a986-f5fbda0881e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:20.803758 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 867/1735] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:20.807298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153bda1-0826-4afd-a0a9-9b371f7ef7d7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:20.816414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a153bda1-0826-4afd-a0a9-9b371f7ef7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:20.892072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a153bda1-0826-4afd-a0a9-9b371f7ef7d7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:20.920090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:20.921644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:20.922427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:20.923119 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] PATH_INFO: `/v3/projects/05c25fd7281e444fbdee942ff9ece3b6` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:20.928373 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05c25fd7281e444fbdee942ff9ece3b6)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:20.939460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:21.075741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd12332-1d02-4ed7-a986-f5fbda0881e6 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:21.134748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05c25fd7281e444fbdee942ff9ece3b6', 'request_id': 'req-71fdae69-c7ae-4538-95f9-8be74b197a06'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:21.177394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05c25fd7281e444fbdee942ff9ece3b6', 'request_id': 'req-71fdae69-c7ae-4538-95f9-8be74b197a06'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:21.199970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05c25fd7281e444fbdee942ff9ece3b6', 'request_id': 'req-71fdae69-c7ae-4538-95f9-8be74b197a06'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:21.256525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] The token cache is being invalidate because project 05c25fd7281e444fbdee942ff9ece3b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:21.257566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:21.266890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:21.271875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 868/1736] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:20 2022] DELETE /identity/v3/projects/05c25fd7281e444fbdee942ff9ece3b6 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:21.275930 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:21.281866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:21.282929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2c35d-9a38-4286-a4a3-5e8cce4dc7b6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:21.284376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2c35d-9a38-4286-a4a3-5e8cce4dc7b6 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:21.287001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2c35d-9a38-4286-a4a3-5e8cce4dc7b6 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:21.289390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2c35d-9a38-4286-a4a3-5e8cce4dc7b6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:21.304340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b6d2c35d-9a38-4286-a4a3-5e8cce4dc7b6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:21.407205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:21.500640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fdae69-c7ae-4538-95f9-8be74b197a06 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:21.547701 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:21.549245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:21.549847 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:21.550470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] PATH_INFO: `/v3/users/ed531602889c437f9701758b3be180be` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:21.552865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed531602889c437f9701758b3be180be)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:21.556911 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:21.662327 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2c35d-9a38-4286-a4a3-5e8cce4dc7b6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:21.673197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2c35d-9a38-4286-a4a3-5e8cce4dc7b6 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:21.714071 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 869/1737] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:23:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1072 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 16:23:21.759982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed531602889c437f9701758b3be180be'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:21.822906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed531602889c437f9701758b3be180be'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:21.961876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed531602889c437f9701758b3be180be'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:21.979179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed531602889c437f9701758b3be180be'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:22.039217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed531602889c437f9701758b3be180be'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:22.085057 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 869/1738] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:21 2022] DELETE /identity/v3/users/ed531602889c437f9701758b3be180be => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:23.885364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fbe76-d14a-4dab-84bf-ccfcfd296f30 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:23.886013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fbe76-d14a-4dab-84bf-ccfcfd296f30 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:23.886329 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fbe76-d14a-4dab-84bf-ccfcfd296f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:24.009125 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-518fbe76-d14a-4dab-84bf-ccfcfd296f30 None None] MFA Rules not processed for user `e175b8de037d45d48cbddd64e552f6f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:24.019942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-518fbe76-d14a-4dab-84bf-ccfcfd296f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:24.176476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:24.310628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 870/1739] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Sep 21 16:23:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:24.353690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:24.359016 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:24.445012 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0faa2-f0d7-4590-abc5-6429552ed198 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:24.463295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ed454-13c4-41ae-a1b0-cf620ebdeb76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:24.464421 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ed454-13c4-41ae-a1b0-cf620ebdeb76 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:24.466002 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ed454-13c4-41ae-a1b0-cf620ebdeb76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:24.466548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ed454-13c4-41ae-a1b0-cf620ebdeb76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:24.517796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ed454-13c4-41ae-a1b0-cf620ebdeb76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:24.524007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042ed454-13c4-41ae-a1b0-cf620ebdeb76 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:24.563842 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 870/1740] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:26.678753 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fbe76-d14a-4dab-84bf-ccfcfd296f30 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:26.682332 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-518fbe76-d14a-4dab-84bf-ccfcfd296f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:26.866749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ed454-13c4-41ae-a1b0-cf620ebdeb76 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:26.871387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-042ed454-13c4-41ae-a1b0-cf620ebdeb76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:26.971448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fbe76-d14a-4dab-84bf-ccfcfd296f30 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:27.004308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:27.009526 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:27.010545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:27.011300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] PATH_INFO: `/v3/projects/c237ae99818b4d3a8d204cec258c5433` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:27.016930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c237ae99818b4d3a8d204cec258c5433)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:27.027761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:27.084390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ed454-13c4-41ae-a1b0-cf620ebdeb76 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:27.104402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:27.105874 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:27.106352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:27.106788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] PATH_INFO: `/v3/projects/9020bde26ac14bba9d04bba7790e01d5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:27.117709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9020bde26ac14bba9d04bba7790e01d5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:27.120102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:27.250024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9020bde26ac14bba9d04bba7790e01d5', 'request_id': 'req-693ab2fc-5afa-445e-ae51-9c834b20bedd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.258611 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:27.277843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9020bde26ac14bba9d04bba7790e01d5', 'request_id': 'req-693ab2fc-5afa-445e-ae51-9c834b20bedd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.288753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9020bde26ac14bba9d04bba7790e01d5', 'request_id': 'req-693ab2fc-5afa-445e-ae51-9c834b20bedd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.317805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] The token cache is being invalidate because project 9020bde26ac14bba9d04bba7790e01d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:27.320417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.321405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.327673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 871/1741] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:26 2022] DELETE /identity/v3/projects/9020bde26ac14bba9d04bba7790e01d5 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:27.332313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:27.333917 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:27.347350 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c237ae99818b4d3a8d204cec258c5433', 'request_id': 'req-cd793dfc-c779-4764-87d4-5a701b1da341'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.390663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c237ae99818b4d3a8d204cec258c5433', 'request_id': 'req-cd793dfc-c779-4764-87d4-5a701b1da341'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.481754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c237ae99818b4d3a8d204cec258c5433', 'request_id': 'req-cd793dfc-c779-4764-87d4-5a701b1da341'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.548178 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693ab2fc-5afa-445e-ae51-9c834b20bedd admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:27.563690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad68c90-85f9-45b2-b4a1-80ec0cb09e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:27.564351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68c90-85f9-45b2-b4a1-80ec0cb09e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:27.564843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68c90-85f9-45b2-b4a1-80ec0cb09e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:27.565148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68c90-85f9-45b2-b4a1-80ec0cb09e34 admin admin] PATH_INFO: `/v3/users/d39b4d2528b84a57863e57948dc97912` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:27.589915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad68c90-85f9-45b2-b4a1-80ec0cb09e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39b4d2528b84a57863e57948dc97912)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:27.592319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad68c90-85f9-45b2-b4a1-80ec0cb09e34 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:27.608698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] The token cache is being invalidate because project c237ae99818b4d3a8d204cec258c5433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:27.609581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.615532 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.619989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 871/1742] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:26 2022] DELETE /identity/v3/projects/c237ae99818b4d3a8d204cec258c5433 => generated 0 bytes in 942 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:27.627409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:27.632053 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:27.850735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3ad68c90-85f9-45b2-b4a1-80ec0cb09e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39b4d2528b84a57863e57948dc97912'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.880174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3ad68c90-85f9-45b2-b4a1-80ec0cb09e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39b4d2528b84a57863e57948dc97912'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.971472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3ad68c90-85f9-45b2-b4a1-80ec0cb09e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39b4d2528b84a57863e57948dc97912'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:27.980432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3ad68c90-85f9-45b2-b4a1-80ec0cb09e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39b4d2528b84a57863e57948dc97912'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:28.002472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3ad68c90-85f9-45b2-b4a1-80ec0cb09e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39b4d2528b84a57863e57948dc97912'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:28.044343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 872/1743] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:27 2022] DELETE /identity/v3/users/d39b4d2528b84a57863e57948dc97912 => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:28.058928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd793dfc-c779-4764-87d4-5a701b1da341 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:28.137029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:28.398861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:28.400785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:28.401066 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:28.401339 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] PATH_INFO: `/v3/users/da03a393d6ae46228f6cb08eec393450` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:28.424528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da03a393d6ae46228f6cb08eec393450)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:28.426916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:28.566595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3883d-0701-4b7c-871a-b9be8aed1ce8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:28.567046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3883d-0701-4b7c-871a-b9be8aed1ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:28.567432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f3883d-0701-4b7c-871a-b9be8aed1ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:28.615955 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da03a393d6ae46228f6cb08eec393450'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:28.639112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da03a393d6ae46228f6cb08eec393450'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:28.639585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da03a393d6ae46228f6cb08eec393450'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:28.655096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da03a393d6ae46228f6cb08eec393450'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:28.696791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-54f3883d-0701-4b7c-871a-b9be8aed1ce8 None None] MFA Rules not processed for user `2e5204475ce7477f9cfeeffed457a445`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:28.707101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da03a393d6ae46228f6cb08eec393450'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:28.708470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-54f3883d-0701-4b7c-871a-b9be8aed1ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:28.724199 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 872/1744] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:27 2022] DELETE /identity/v3/users/da03a393d6ae46228f6cb08eec393450 => generated 0 bytes in 1095 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:29.033777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 873/1745] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:29.383988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:29.385940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:29.725434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46879ac4-384f-45ca-afa6-fb5726d49096 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:29.757080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166ff37d-ead3-4420-b11e-eaf2a3572f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:29.762738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ff37d-ead3-4420-b11e-eaf2a3572f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:29.764675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ff37d-ead3-4420-b11e-eaf2a3572f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:29.766521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166ff37d-ead3-4420-b11e-eaf2a3572f47 admin admin] PATH_INFO: `/v3/projects/1a133c2bd62543b2b73f49227ad54605` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:29.782235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ff37d-ead3-4420-b11e-eaf2a3572f47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a133c2bd62543b2b73f49227ad54605)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:29.787441 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166ff37d-ead3-4420-b11e-eaf2a3572f47 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:29.932192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:29.993586 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-166ff37d-ead3-4420-b11e-eaf2a3572f47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a133c2bd62543b2b73f49227ad54605', 'request_id': 'req-166ff37d-ead3-4420-b11e-eaf2a3572f47'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:30.010802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-166ff37d-ead3-4420-b11e-eaf2a3572f47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a133c2bd62543b2b73f49227ad54605', 'request_id': 'req-166ff37d-ead3-4420-b11e-eaf2a3572f47'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:30.021540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-166ff37d-ead3-4420-b11e-eaf2a3572f47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a133c2bd62543b2b73f49227ad54605', 'request_id': 'req-166ff37d-ead3-4420-b11e-eaf2a3572f47'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:30.085205 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-166ff37d-ead3-4420-b11e-eaf2a3572f47 admin admin] The token cache is being invalidate because project 1a133c2bd62543b2b73f49227ad54605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:30.085517 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-166ff37d-ead3-4420-b11e-eaf2a3572f47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:30.096831 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-166ff37d-ead3-4420-b11e-eaf2a3572f47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:30.101377 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 873/1746] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:29 2022] DELETE /identity/v3/projects/1a133c2bd62543b2b73f49227ad54605 => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:30.109607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c29a0b-514d-476b-af29-9cc9d314554c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:30.110340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c29a0b-514d-476b-af29-9cc9d314554c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:30.110340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c29a0b-514d-476b-af29-9cc9d314554c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:30.130643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:30.167401 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-60c29a0b-514d-476b-af29-9cc9d314554c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:30.169087 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-60c29a0b-514d-476b-af29-9cc9d314554c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:30.441918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 874/1747] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:23:30.465220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c464a8-87ce-4de3-9d7a-fa70b6666165 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:30.468155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c464a8-87ce-4de3-9d7a-fa70b6666165 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:30.470160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c464a8-87ce-4de3-9d7a-fa70b6666165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:30.543033 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f5c464a8-87ce-4de3-9d7a-fa70b6666165 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:30.545430 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f5c464a8-87ce-4de3-9d7a-fa70b6666165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:30.595995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 874/1748] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:30.603571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c29a0b-514d-476b-af29-9cc9d314554c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:30.645102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c29a0b-514d-476b-af29-9cc9d314554c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:30.672733 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31e7e9-ccaa-4799-932a-52487665fced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:30.674335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31e7e9-ccaa-4799-932a-52487665fced admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:30.675725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31e7e9-ccaa-4799-932a-52487665fced admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:30.676436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31e7e9-ccaa-4799-932a-52487665fced admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:30.679528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d31e7e9-ccaa-4799-932a-52487665fced admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:23:30.680489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d31e7e9-ccaa-4799-932a-52487665fced admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:30.685998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d31e7e9-ccaa-4799-932a-52487665fced admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:30.716902 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 875/1749] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:23:30.723615 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c464a8-87ce-4de3-9d7a-fa70b6666165 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:30.742514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c464a8-87ce-4de3-9d7a-fa70b6666165 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:30.765320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ef76e-09a4-4bb5-be31-619298eb978c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:30.766982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ef76e-09a4-4bb5-be31-619298eb978c admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:30.768060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ef76e-09a4-4bb5-be31-619298eb978c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:30.768700 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ef76e-09a4-4bb5-be31-619298eb978c admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:30.769621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ef76e-09a4-4bb5-be31-619298eb978c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:30.773724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26ef76e-09a4-4bb5-be31-619298eb978c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:30.894958 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 875/1750] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:30 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:30.904193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31e7e9-ccaa-4799-932a-52487665fced admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:30.965510 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31e7e9-ccaa-4799-932a-52487665fced admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:31.194211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791a308-ee23-41b3-8ed1-44c16a0a10ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:31.195334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791a308-ee23-41b3-8ed1-44c16a0a10ba admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:31.195334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791a308-ee23-41b3-8ed1-44c16a0a10ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:31.195684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791a308-ee23-41b3-8ed1-44c16a0a10ba admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:31.197015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791a308-ee23-41b3-8ed1-44c16a0a10ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:31.199357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791a308-ee23-41b3-8ed1-44c16a0a10ba admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:31.256475 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 876/1751] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:30 2022] POST /identity/v3/users => generated 287 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:31.259410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ef76e-09a4-4bb5-be31-619298eb978c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:31.300600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ef76e-09a4-4bb5-be31-619298eb978c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:31.337114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197adcd-5deb-4dff-bbe9-f725097bd1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:31.341092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197adcd-5deb-4dff-bbe9-f725097bd1ef admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:31.341891 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197adcd-5deb-4dff-bbe9-f725097bd1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:31.346456 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197adcd-5deb-4dff-bbe9-f725097bd1ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:31.348462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197adcd-5deb-4dff-bbe9-f725097bd1ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:31.352111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197adcd-5deb-4dff-bbe9-f725097bd1ef admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:31.407681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 876/1752] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:31.410463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791a308-ee23-41b3-8ed1-44c16a0a10ba admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:31.442617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791a308-ee23-41b3-8ed1-44c16a0a10ba admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:31.459156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e444d-250c-499a-b437-74a7972304de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:31.460815 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e444d-250c-499a-b437-74a7972304de admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:31.461540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e444d-250c-499a-b437-74a7972304de admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:31.462314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e444d-250c-499a-b437-74a7972304de admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:31.465117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e444d-250c-499a-b437-74a7972304de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:31.470158 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e444d-250c-499a-b437-74a7972304de admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:31.531693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 877/1753] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:31.540593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197adcd-5deb-4dff-bbe9-f725097bd1ef admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:31.578741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197adcd-5deb-4dff-bbe9-f725097bd1ef admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:31.645905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3713548-b28b-42a7-8427-1a61b0934518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:31.647838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3713548-b28b-42a7-8427-1a61b0934518 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:31.648952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3713548-b28b-42a7-8427-1a61b0934518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:31.650901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3713548-b28b-42a7-8427-1a61b0934518 admin admin] PATH_INFO: `/v3/projects/2e28ffc92a704580aa12641b62d0d404/users/f6256e2247594d34a80f7a3b133104cb/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:31.705854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3713548-b28b-42a7-8427-1a61b0934518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e28ffc92a704580aa12641b62d0d404, user_id=f6256e2247594d34a80f7a3b133104cb, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:31.714810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3713548-b28b-42a7-8427-1a61b0934518 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:31.755631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 877/1754] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:31 2022] PUT /identity/v3/projects/2e28ffc92a704580aa12641b62d0d404/users/f6256e2247594d34a80f7a3b133104cb/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:31.777991 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace3b65-a807-4aba-9221-146f30db08c1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:31.779321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace3b65-a807-4aba-9221-146f30db08c1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:31.779321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace3b65-a807-4aba-9221-146f30db08c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:31.891358 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5ace3b65-a807-4aba-9221-146f30db08c1 None None] MFA Rules not processed for user `f6256e2247594d34a80f7a3b133104cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:31.892789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5ace3b65-a807-4aba-9221-146f30db08c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:32.164136 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:32.182004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 878/1755] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:23:32.190381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3713548-b28b-42a7-8427-1a61b0934518 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:32.486087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3713548-b28b-42a7-8427-1a61b0934518 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:32.500042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8c0b5-558c-4bf8-898b-8db753900fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:32.537087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8c0b5-558c-4bf8-898b-8db753900fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:32.537985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8c0b5-558c-4bf8-898b-8db753900fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:32.539271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8c0b5-558c-4bf8-898b-8db753900fa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:32.544855 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8c0b5-558c-4bf8-898b-8db753900fa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:32.551546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8c0b5-558c-4bf8-898b-8db753900fa4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:32.771403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 878/1756] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:32 2022] POST /identity/v3/projects => generated 373 bytes in 583 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:32.793666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace3b65-a807-4aba-9221-146f30db08c1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:32.937413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace3b65-a807-4aba-9221-146f30db08c1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:33.255669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ba068-ca99-42a8-8c00-a3cf8c6f61bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:33.259108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ba068-ca99-42a8-8c00-a3cf8c6f61bc admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:33.259981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ba068-ca99-42a8-8c00-a3cf8c6f61bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:33.260746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ba068-ca99-42a8-8c00-a3cf8c6f61bc admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:33.262297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ba068-ca99-42a8-8c00-a3cf8c6f61bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:33.269832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636ba068-ca99-42a8-8c00-a3cf8c6f61bc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:33.275615 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687c712-afed-40be-b376-2db4940a5dee None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:33.275904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687c712-afed-40be-b376-2db4940a5dee None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:33.276156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687c712-afed-40be-b376-2db4940a5dee None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:33.278903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 879/1757] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:23:33.352639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 879/1758] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:32 2022] POST /identity/v3/users => generated 293 bytes in 575 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:33.356088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687c712-afed-40be-b376-2db4940a5dee None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:33.390481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc861dd-e190-47ac-b0c0-97bf442054d8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:33.391219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc861dd-e190-47ac-b0c0-97bf442054d8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:33.391695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc861dd-e190-47ac-b0c0-97bf442054d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:33.412017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687c712-afed-40be-b376-2db4940a5dee None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:33.439163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47512e7d-f2f5-43be-b81d-0fc9f31e8428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:33.441621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47512e7d-f2f5-43be-b81d-0fc9f31e8428 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:33.442148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47512e7d-f2f5-43be-b81d-0fc9f31e8428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:33.446455 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47512e7d-f2f5-43be-b81d-0fc9f31e8428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:33.450521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47512e7d-f2f5-43be-b81d-0fc9f31e8428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:33.455908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47512e7d-f2f5-43be-b81d-0fc9f31e8428 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:33.468695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7cc861dd-e190-47ac-b0c0-97bf442054d8 None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:33.477364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7cc861dd-e190-47ac-b0c0-97bf442054d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:33.522534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 880/1759] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:33.523793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47512e7d-f2f5-43be-b81d-0fc9f31e8428 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:33.544665 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47512e7d-f2f5-43be-b81d-0fc9f31e8428 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:33.556314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ded92-185c-4aac-96ce-41116592f5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:33.558118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ded92-185c-4aac-96ce-41116592f5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:33.558591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ded92-185c-4aac-96ce-41116592f5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:33.559012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ded92-185c-4aac-96ce-41116592f5e0 admin admin] PATH_INFO: `/v3/projects/eebcb4626f7c4c4aaf70c1f8dfac2f4c/users/698277dfe4ae4f53823a6a9cb41e77ad/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:33.597584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ded92-185c-4aac-96ce-41116592f5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eebcb4626f7c4c4aaf70c1f8dfac2f4c, user_id=698277dfe4ae4f53823a6a9cb41e77ad, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:33.601065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ded92-185c-4aac-96ce-41116592f5e0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:33.629921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 881/1760] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:33 2022] PUT /identity/v3/projects/eebcb4626f7c4c4aaf70c1f8dfac2f4c/users/698277dfe4ae4f53823a6a9cb41e77ad/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:33.635671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ded92-185c-4aac-96ce-41116592f5e0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:33.657396 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:33.806791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 880/1761] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:23:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:33.889715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ded92-185c-4aac-96ce-41116592f5e0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:33.901812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550f8f2-f35a-461a-ac37-12f56ec9dfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:33.904339 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550f8f2-f35a-461a-ac37-12f56ec9dfc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:33.904874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550f8f2-f35a-461a-ac37-12f56ec9dfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:33.905499 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550f8f2-f35a-461a-ac37-12f56ec9dfc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:33.915562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550f8f2-f35a-461a-ac37-12f56ec9dfc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:33.943346 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550f8f2-f35a-461a-ac37-12f56ec9dfc0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:33.991269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc861dd-e190-47ac-b0c0-97bf442054d8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:34.001122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7cc861dd-e190-47ac-b0c0-97bf442054d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:34.046390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 882/1762] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:23:34.051771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550f8f2-f35a-461a-ac37-12f56ec9dfc0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:34.059006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc861dd-e190-47ac-b0c0-97bf442054d8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:34.083670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913b725-c28f-4687-bbdd-ebcfff45ebe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:34.083670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913b725-c28f-4687-bbdd-ebcfff45ebe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:34.083670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913b725-c28f-4687-bbdd-ebcfff45ebe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:34.086530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913b725-c28f-4687-bbdd-ebcfff45ebe7 admin admin] PATH_INFO: `/v3/projects/80e96a43533f4c299ab83a58a3ae2234` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:34.099192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913b725-c28f-4687-bbdd-ebcfff45ebe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80e96a43533f4c299ab83a58a3ae2234)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:34.101770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913b725-c28f-4687-bbdd-ebcfff45ebe7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:34.106013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550f8f2-f35a-461a-ac37-12f56ec9dfc0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:34.148464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af19659-99ef-4e93-b7f0-f7a8074a9cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:34.149592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af19659-99ef-4e93-b7f0-f7a8074a9cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:34.150130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af19659-99ef-4e93-b7f0-f7a8074a9cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:34.150581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af19659-99ef-4e93-b7f0-f7a8074a9cf4 admin admin] PATH_INFO: `/v3/projects/eebcb4626f7c4c4aaf70c1f8dfac2f4c/users/698277dfe4ae4f53823a6a9cb41e77ad/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:34.167270 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af19659-99ef-4e93-b7f0-f7a8074a9cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eebcb4626f7c4c4aaf70c1f8dfac2f4c, user_id=698277dfe4ae4f53823a6a9cb41e77ad, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:34.176389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af19659-99ef-4e93-b7f0-f7a8074a9cf4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:34.258238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 883/1763] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:34 2022] PUT /identity/v3/projects/eebcb4626f7c4c4aaf70c1f8dfac2f4c/users/698277dfe4ae4f53823a6a9cb41e77ad/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:34.259977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af19659-99ef-4e93-b7f0-f7a8074a9cf4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:34.266114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4af19659-99ef-4e93-b7f0-f7a8074a9cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:34.304592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2913b725-c28f-4687-bbdd-ebcfff45ebe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80e96a43533f4c299ab83a58a3ae2234', 'request_id': 'req-2913b725-c28f-4687-bbdd-ebcfff45ebe7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:34.346018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2913b725-c28f-4687-bbdd-ebcfff45ebe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80e96a43533f4c299ab83a58a3ae2234', 'request_id': 'req-2913b725-c28f-4687-bbdd-ebcfff45ebe7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:34.375817 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2913b725-c28f-4687-bbdd-ebcfff45ebe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80e96a43533f4c299ab83a58a3ae2234', 'request_id': 'req-2913b725-c28f-4687-bbdd-ebcfff45ebe7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:34.428272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2913b725-c28f-4687-bbdd-ebcfff45ebe7 admin admin] The token cache is being invalidate because project 80e96a43533f4c299ab83a58a3ae2234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:34.428803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2913b725-c28f-4687-bbdd-ebcfff45ebe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:34.437146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2913b725-c28f-4687-bbdd-ebcfff45ebe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:34.443410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 881/1764] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:33 2022] DELETE /identity/v3/projects/80e96a43533f4c299ab83a58a3ae2234 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:34.454236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd055729-14de-48e6-9be8-765ee5844ce6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:34.454741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd055729-14de-48e6-9be8-765ee5844ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:34.455238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd055729-14de-48e6-9be8-765ee5844ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:34.514095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:34.633555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bd055729-14de-48e6-9be8-765ee5844ce6 None None] MFA Rules not processed for user `698277dfe4ae4f53823a6a9cb41e77ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:34.636223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bd055729-14de-48e6-9be8-765ee5844ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:34.648767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af19659-99ef-4e93-b7f0-f7a8074a9cf4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:34.994824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:34.998914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:35.000186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:35.000733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] PATH_INFO: `/v3/projects/d8a214c2013b4a1c8e129a0e33a6aa0b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:35.003869 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a214c2013b4a1c8e129a0e33a6aa0b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:35.011206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:35.188991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:35.189591 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 882/1765] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 746 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:35.193973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72e1dc-12f2-4629-b566-311e5f16ab70 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:35.194692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72e1dc-12f2-4629-b566-311e5f16ab70 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:35.195880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72e1dc-12f2-4629-b566-311e5f16ab70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:35.227914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a214c2013b4a1c8e129a0e33a6aa0b', 'request_id': 'req-879472ad-014f-42cb-8de7-8372e2753e7f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:35.271604 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a214c2013b4a1c8e129a0e33a6aa0b', 'request_id': 'req-879472ad-014f-42cb-8de7-8372e2753e7f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:35.301805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a214c2013b4a1c8e129a0e33a6aa0b', 'request_id': 'req-879472ad-014f-42cb-8de7-8372e2753e7f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:35.311224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1e72e1dc-12f2-4629-b566-311e5f16ab70 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:35.317051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1e72e1dc-12f2-4629-b566-311e5f16ab70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:35.375064 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] The token cache is being invalidate because project d8a214c2013b4a1c8e129a0e33a6aa0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:35.383226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:35.391420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:35.404883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 884/1766] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:34 2022] DELETE /identity/v3/projects/d8a214c2013b4a1c8e129a0e33a6aa0b => generated 0 bytes in 1146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:35.406887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:35.416469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:35.777805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 883/1767] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 586 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:35.780425 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d61afc-d3b2-403a-a884-a3cacd839e27 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:35.781936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d61afc-d3b2-403a-a884-a3cacd839e27 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:35.784112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d61afc-d3b2-403a-a884-a3cacd839e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:35.832638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879472ad-014f-42cb-8de7-8372e2753e7f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:35.861594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59b633-8d08-43ce-890c-a7e5206d6239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:35.863187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59b633-8d08-43ce-890c-a7e5206d6239 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:35.864038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59b633-8d08-43ce-890c-a7e5206d6239 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:35.864798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59b633-8d08-43ce-890c-a7e5206d6239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:35.874970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cd59b633-8d08-43ce-890c-a7e5206d6239 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:35.896246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-35d61afc-d3b2-403a-a884-a3cacd839e27 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:35.916027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-35d61afc-d3b2-403a-a884-a3cacd839e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:35.988484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 884/1768] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:35.989159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d61afc-d3b2-403a-a884-a3cacd839e27 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:36.003291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-35d61afc-d3b2-403a-a884-a3cacd839e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:36.243239 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59b633-8d08-43ce-890c-a7e5206d6239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:36.247885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd59b633-8d08-43ce-890c-a7e5206d6239 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:36.282201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 885/1769] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 877 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:36.285974 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98bcd-9f9d-45e6-94fe-e471fabe72fb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:36.286754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98bcd-9f9d-45e6-94fe-e471fabe72fb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:36.287046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a98bcd-9f9d-45e6-94fe-e471fabe72fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:36.320207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d61afc-d3b2-403a-a884-a3cacd839e27 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:36.342207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce53d8f-9270-4eef-a601-5f64ab82bc98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:36.342967 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce53d8f-9270-4eef-a601-5f64ab82bc98 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:36.343115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce53d8f-9270-4eef-a601-5f64ab82bc98 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:36.343368 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce53d8f-9270-4eef-a601-5f64ab82bc98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:36.349000 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e9a98bcd-9f9d-45e6-94fe-e471fabe72fb None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:36.353376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e9a98bcd-9f9d-45e6-94fe-e471fabe72fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:36.363211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8ce53d8f-9270-4eef-a601-5f64ab82bc98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:36.384348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 886/1770] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:36.387843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b21ae9-290d-4d54-958e-35b9f80ed4e1 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:36.388242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b21ae9-290d-4d54-958e-35b9f80ed4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:36.388536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b21ae9-290d-4d54-958e-35b9f80ed4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:36.417174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce53d8f-9270-4eef-a601-5f64ab82bc98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:36.421742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce53d8f-9270-4eef-a601-5f64ab82bc98 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:36.458142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 885/1771] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:36.459246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce53d8f-9270-4eef-a601-5f64ab82bc98 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:36.468437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-10b21ae9-290d-4d54-958e-35b9f80ed4e1 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:36.469832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-10b21ae9-290d-4d54-958e-35b9f80ed4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:36.508445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce53d8f-9270-4eef-a601-5f64ab82bc98 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:36.516107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 887/1772] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:36.517217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b21ae9-290d-4d54-958e-35b9f80ed4e1 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:36.528567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317755e-4010-4604-adde-5aab6a6fa379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:36.531624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317755e-4010-4604-adde-5aab6a6fa379 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:36.533086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317755e-4010-4604-adde-5aab6a6fa379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:36.538519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317755e-4010-4604-adde-5aab6a6fa379 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:36.540707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317755e-4010-4604-adde-5aab6a6fa379 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:23:36.548612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317755e-4010-4604-adde-5aab6a6fa379 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:36.564811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b21ae9-290d-4d54-958e-35b9f80ed4e1 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:36.568479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317755e-4010-4604-adde-5aab6a6fa379 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:36.585911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5a38e-3ef2-4c5a-8b01-0feb58bc6032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:36.586587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a38e-3ef2-4c5a-8b01-0feb58bc6032 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:36.586810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:36.587010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a38e-3ef2-4c5a-8b01-0feb58bc6032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:36.587206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5a38e-3ef2-4c5a-8b01-0feb58bc6032 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:36.587791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5a38e-3ef2-4c5a-8b01-0feb58bc6032 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:23:36.588139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5a38e-3ef2-4c5a-8b01-0feb58bc6032 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:36.596996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5a38e-3ef2-4c5a-8b01-0feb58bc6032 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:36.651823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 888/1773] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:23:36.659174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5a38e-3ef2-4c5a-8b01-0feb58bc6032 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:36.674374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 886/1774] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:23:36.692501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317755e-4010-4604-adde-5aab6a6fa379 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:36.703347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5a38e-3ef2-4c5a-8b01-0feb58bc6032 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:36.718659 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561a679-c8c6-458d-8df9-8af6172d939d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:36.721392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561a679-c8c6-458d-8df9-8af6172d939d admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:36.726862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561a679-c8c6-458d-8df9-8af6172d939d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:36.728014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5561a679-c8c6-458d-8df9-8af6172d939d admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:36.732262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561a679-c8c6-458d-8df9-8af6172d939d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:36.743451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5561a679-c8c6-458d-8df9-8af6172d939d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:36.780381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317755e-4010-4604-adde-5aab6a6fa379 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:36.808384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11d95a-03e0-4115-9422-340f7b93a9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:36.809632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11d95a-03e0-4115-9422-340f7b93a9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:36.812084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11d95a-03e0-4115-9422-340f7b93a9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:36.812638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11d95a-03e0-4115-9422-340f7b93a9f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:36.815127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11d95a-03e0-4115-9422-340f7b93a9f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:36.834029 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11d95a-03e0-4115-9422-340f7b93a9f2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:36.873246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 889/1775] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:36 2022] POST /identity/v3/projects => generated 383 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:36.875437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561a679-c8c6-458d-8df9-8af6172d939d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:37.018543 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:37.040707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 887/1776] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:36 2022] POST /identity/v3/projects => generated 379 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:37.044884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11d95a-03e0-4115-9422-340f7b93a9f2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:37.127040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5561a679-c8c6-458d-8df9-8af6172d939d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:37.299874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff939be3-0e6e-4c02-a403-725e0be42dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:37.302203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff939be3-0e6e-4c02-a403-725e0be42dff admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:37.303304 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff939be3-0e6e-4c02-a403-725e0be42dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:37.306324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff939be3-0e6e-4c02-a403-725e0be42dff admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:37.308106 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff939be3-0e6e-4c02-a403-725e0be42dff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:37.312905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff939be3-0e6e-4c02-a403-725e0be42dff admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:37.370877 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 890/1777] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:36 2022] POST /identity/v3/users => generated 292 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:37.371911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff939be3-0e6e-4c02-a403-725e0be42dff admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:37.398020 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff939be3-0e6e-4c02-a403-725e0be42dff admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:37.420136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33b69d-53c9-4221-a223-47342f9d89e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:37.423063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33b69d-53c9-4221-a223-47342f9d89e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:37.423436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33b69d-53c9-4221-a223-47342f9d89e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:37.423770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33b69d-53c9-4221-a223-47342f9d89e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:37.424612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33b69d-53c9-4221-a223-47342f9d89e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:37.431036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be33b69d-53c9-4221-a223-47342f9d89e5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:37.466223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 891/1778] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:37.467723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33b69d-53c9-4221-a223-47342f9d89e5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:37.498954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11d95a-03e0-4115-9422-340f7b93a9f2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:37.536213 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3abd8-fa7c-49e8-9eb3-cdac9f21ccaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:37.539614 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3abd8-fa7c-49e8-9eb3-cdac9f21ccaa admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:37.545723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33b69d-53c9-4221-a223-47342f9d89e5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:37.554417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3abd8-fa7c-49e8-9eb3-cdac9f21ccaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:37.555540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3abd8-fa7c-49e8-9eb3-cdac9f21ccaa admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:37.558457 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3abd8-fa7c-49e8-9eb3-cdac9f21ccaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:37.564162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3abd8-fa7c-49e8-9eb3-cdac9f21ccaa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:37.565104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44c53c-c070-4f21-95e2-fb0a6d6061bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:37.575016 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44c53c-c070-4f21-95e2-fb0a6d6061bd admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:37.580069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44c53c-c070-4f21-95e2-fb0a6d6061bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:37.580584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44c53c-c070-4f21-95e2-fb0a6d6061bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:37.594424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44c53c-c070-4f21-95e2-fb0a6d6061bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:37.609607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44c53c-c070-4f21-95e2-fb0a6d6061bd admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:37.640597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 888/1779] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:37 2022] POST /identity/v3/users => generated 290 bytes in 596 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:37.647001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3abd8-fa7c-49e8-9eb3-cdac9f21ccaa admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:37.657937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e0c3abd8-fa7c-49e8-9eb3-cdac9f21ccaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:37.683680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 892/1780] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:37.684798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44c53c-c070-4f21-95e2-fb0a6d6061bd admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:37.718309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44c53c-c070-4f21-95e2-fb0a6d6061bd admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:37.743002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb5e6b-4560-46e0-a624-a5e498e95f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:37.743250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3abd8-fa7c-49e8-9eb3-cdac9f21ccaa admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:37.743705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb5e6b-4560-46e0-a624-a5e498e95f29 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:37.743994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb5e6b-4560-46e0-a624-a5e498e95f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:37.744269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb5e6b-4560-46e0-a624-a5e498e95f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:37.745140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb5e6b-4560-46e0-a624-a5e498e95f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:37.759129 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb5e6b-4560-46e0-a624-a5e498e95f29 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:37.761382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798c5ed-56a5-4bfe-a256-8520c5884b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:37.768127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798c5ed-56a5-4bfe-a256-8520c5884b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:37.768808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798c5ed-56a5-4bfe-a256-8520c5884b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:37.773565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798c5ed-56a5-4bfe-a256-8520c5884b29 admin admin] PATH_INFO: `/v3/users/f6256e2247594d34a80f7a3b133104cb` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:37.777318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798c5ed-56a5-4bfe-a256-8520c5884b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6256e2247594d34a80f7a3b133104cb)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:37.786071 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798c5ed-56a5-4bfe-a256-8520c5884b29 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:37.802408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 893/1781] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:37.816565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb5e6b-4560-46e0-a624-a5e498e95f29 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:37.856454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb5e6b-4560-46e0-a624-a5e498e95f29 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:37.903884 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504bd05-5974-4543-bee9-e8ff6910b64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:37.904671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504bd05-5974-4543-bee9-e8ff6910b64b admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:37.905124 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504bd05-5974-4543-bee9-e8ff6910b64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:37.905957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504bd05-5974-4543-bee9-e8ff6910b64b admin admin] PATH_INFO: `/v3/projects/5d14034d38f4450090b2c529bd45b4b9/users/17b3a27f3e364de0a30e9dd046ae8b4d/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:37.948802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e798c5ed-56a5-4bfe-a256-8520c5884b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6256e2247594d34a80f7a3b133104cb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:37.966809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e798c5ed-56a5-4bfe-a256-8520c5884b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6256e2247594d34a80f7a3b133104cb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:37.967207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504bd05-5974-4543-bee9-e8ff6910b64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d14034d38f4450090b2c529bd45b4b9, user_id=17b3a27f3e364de0a30e9dd046ae8b4d, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:37.972515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504bd05-5974-4543-bee9-e8ff6910b64b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:38.024905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e798c5ed-56a5-4bfe-a256-8520c5884b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6256e2247594d34a80f7a3b133104cb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:38.031105 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e798c5ed-56a5-4bfe-a256-8520c5884b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6256e2247594d34a80f7a3b133104cb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:38.033273 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 894/1782] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:37 2022] PUT /identity/v3/projects/5d14034d38f4450090b2c529bd45b4b9/users/17b3a27f3e364de0a30e9dd046ae8b4d/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:38.044698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504bd05-5974-4543-bee9-e8ff6910b64b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:38.053252 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0504bd05-5974-4543-bee9-e8ff6910b64b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:38.071883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e798c5ed-56a5-4bfe-a256-8520c5884b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6256e2247594d34a80f7a3b133104cb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:38.097116 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 889/1783] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:37 2022] DELETE /identity/v3/users/f6256e2247594d34a80f7a3b133104cb => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:38.100114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3534310-e92a-4ab1-a0c6-72145cfed7e2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:38.108447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3534310-e92a-4ab1-a0c6-72145cfed7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:38.109282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3534310-e92a-4ab1-a0c6-72145cfed7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:38.264698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c3534310-e92a-4ab1-a0c6-72145cfed7e2 None None] MFA Rules not processed for user `17b3a27f3e364de0a30e9dd046ae8b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:38.266527 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c3534310-e92a-4ab1-a0c6-72145cfed7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:38.337749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:38.427263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504bd05-5974-4543-bee9-e8ff6910b64b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:38.462189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:38.533449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 890/1784] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:38.539177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839dfc-9421-4f18-89c1-bf79ab518859 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:38.542949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839dfc-9421-4f18-89c1-bf79ab518859 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:38.543276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839dfc-9421-4f18-89c1-bf79ab518859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:38.577484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-80839dfc-9421-4f18-89c1-bf79ab518859 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:38.579526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-80839dfc-9421-4f18-89c1-bf79ab518859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:38.727486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c650d-9b0c-4b6c-9922-ad0199139aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:38.728778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c650d-9b0c-4b6c-9922-ad0199139aed admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:38.729785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c650d-9b0c-4b6c-9922-ad0199139aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:38.730371 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c650d-9b0c-4b6c-9922-ad0199139aed admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:38.731720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c650d-9b0c-4b6c-9922-ad0199139aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:38.747132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34c650d-9b0c-4b6c-9922-ad0199139aed admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:38.795759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 895/1785] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 756 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:38.797726 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c650d-9b0c-4b6c-9922-ad0199139aed admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:38.807812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f34c650d-9b0c-4b6c-9922-ad0199139aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:38.860036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 891/1786] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:38.869270 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839dfc-9421-4f18-89c1-bf79ab518859 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:38.924005 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c650d-9b0c-4b6c-9922-ad0199139aed admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:38.943574 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7d317-60d5-40c7-b1c0-ef400c783ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:38.948356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839dfc-9421-4f18-89c1-bf79ab518859 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:38.948820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7d317-60d5-40c7-b1c0-ef400c783ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:38.952672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7d317-60d5-40c7-b1c0-ef400c783ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:38.953076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d7d317-60d5-40c7-b1c0-ef400c783ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:38.957010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7d317-60d5-40c7-b1c0-ef400c783ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:38.972499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da93681-6ffa-4c8f-90d3-6b7bf450436a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:38.975362 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da93681-6ffa-4c8f-90d3-6b7bf450436a admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:38.977158 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da93681-6ffa-4c8f-90d3-6b7bf450436a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:38.978203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da93681-6ffa-4c8f-90d3-6b7bf450436a admin admin] PATH_INFO: `/v3/projects/dffcc69d970c4c6188311cb104fd9964/users/81c831e716d544e885d951de805edccc/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:38.985532 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d7d317-60d5-40c7-b1c0-ef400c783ab3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:39.113431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da93681-6ffa-4c8f-90d3-6b7bf450436a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dffcc69d970c4c6188311cb104fd9964, user_id=81c831e716d544e885d951de805edccc, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:39.141400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da93681-6ffa-4c8f-90d3-6b7bf450436a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:39.253376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 896/1787] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:38 2022] POST /identity/v3/projects => generated 383 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:39.272990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7d317-60d5-40c7-b1c0-ef400c783ab3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:39.290946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-66d7d317-60d5-40c7-b1c0-ef400c783ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:39.310479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 892/1788] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:38 2022] PUT /identity/v3/projects/dffcc69d970c4c6188311cb104fd9964/users/81c831e716d544e885d951de805edccc/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:39.319551 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da93681-6ffa-4c8f-90d3-6b7bf450436a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:39.617251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d7d317-60d5-40c7-b1c0-ef400c783ab3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:39.633594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551af0d8-b378-4eaf-9a98-f688acaf26b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:39.634968 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551af0d8-b378-4eaf-9a98-f688acaf26b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:39.635362 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551af0d8-b378-4eaf-9a98-f688acaf26b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:39.635836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551af0d8-b378-4eaf-9a98-f688acaf26b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:39.641477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da93681-6ffa-4c8f-90d3-6b7bf450436a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:39.880739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:39.951957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551af0d8-b378-4eaf-9a98-f688acaf26b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:39.958950 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b66d4-9b0d-4121-a70e-d1404b446ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:39.961612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b66d4-9b0d-4121-a70e-d1404b446ceb admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:39.962268 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b66d4-9b0d-4121-a70e-d1404b446ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:39.962894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b66d4-9b0d-4121-a70e-d1404b446ceb admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:39.966796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551af0d8-b378-4eaf-9a98-f688acaf26b6 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:39.977472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b66d4-9b0d-4121-a70e-d1404b446ceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:39.992437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b66d4-9b0d-4121-a70e-d1404b446ceb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:40.024519 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 897/1789] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:40.028013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997b3ba-0ee7-41fa-90a5-6efcbf5acade None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:40.034200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997b3ba-0ee7-41fa-90a5-6efcbf5acade None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:40.034905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997b3ba-0ee7-41fa-90a5-6efcbf5acade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:40.078103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 893/1790] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:39 2022] POST /identity/v3/users => generated 292 bytes in 766 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:40.083951 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b66d4-9b0d-4121-a70e-d1404b446ceb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:40.118199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b66d4-9b0d-4121-a70e-d1404b446ceb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:40.134437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14be1d9-0784-4368-8f45-f69eef6e12c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:40.137139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14be1d9-0784-4368-8f45-f69eef6e12c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:40.138028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14be1d9-0784-4368-8f45-f69eef6e12c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:40.138978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14be1d9-0784-4368-8f45-f69eef6e12c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:40.140831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14be1d9-0784-4368-8f45-f69eef6e12c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:40.144796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14be1d9-0784-4368-8f45-f69eef6e12c8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:40.186792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 894/1791] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:40.193536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14be1d9-0784-4368-8f45-f69eef6e12c8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:40.206935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e997b3ba-0ee7-41fa-90a5-6efcbf5acade None None] MFA Rules not processed for user `81c831e716d544e885d951de805edccc`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:40.210868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e997b3ba-0ee7-41fa-90a5-6efcbf5acade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:40.226361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14be1d9-0784-4368-8f45-f69eef6e12c8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:40.236064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15300283-3fe5-4359-8ed8-0b3f2bd4fb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:40.237523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15300283-3fe5-4359-8ed8-0b3f2bd4fb36 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:40.238241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15300283-3fe5-4359-8ed8-0b3f2bd4fb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:40.238839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15300283-3fe5-4359-8ed8-0b3f2bd4fb36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:40.243771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15300283-3fe5-4359-8ed8-0b3f2bd4fb36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:40.251483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15300283-3fe5-4359-8ed8-0b3f2bd4fb36 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:40.314496 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 895/1792] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:40.324258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15300283-3fe5-4359-8ed8-0b3f2bd4fb36 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:40.349188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:40.375192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15300283-3fe5-4359-8ed8-0b3f2bd4fb36 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:40.390928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fa58e-bb50-4ff4-bb88-50e97c7891e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:40.392053 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fa58e-bb50-4ff4-bb88-50e97c7891e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:40.392521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fa58e-bb50-4ff4-bb88-50e97c7891e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:40.392990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fa58e-bb50-4ff4-bb88-50e97c7891e0 admin admin] PATH_INFO: `/v3/projects/c8ee5ae41ae748fe8a2604c01415ecec/users/041ac6a41f814755af6874f12bfdd960/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:40.483666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fa58e-bb50-4ff4-bb88-50e97c7891e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ee5ae41ae748fe8a2604c01415ecec, user_id=041ac6a41f814755af6874f12bfdd960, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:40.505366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fa58e-bb50-4ff4-bb88-50e97c7891e0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:40.570819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 896/1793] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:40 2022] PUT /identity/v3/projects/c8ee5ae41ae748fe8a2604c01415ecec/users/041ac6a41f814755af6874f12bfdd960/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:40.583852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b412436-fdf0-4b2a-839b-8d22eeb6f25c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:40.584454 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b412436-fdf0-4b2a-839b-8d22eeb6f25c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:40.584772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b412436-fdf0-4b2a-839b-8d22eeb6f25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:40.590376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 898/1794] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 565 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:40.592083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997b3ba-0ee7-41fa-90a5-6efcbf5acade None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:40.667737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997b3ba-0ee7-41fa-90a5-6efcbf5acade None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:40.695831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0b412436-fdf0-4b2a-839b-8d22eeb6f25c None None] MFA Rules not processed for user `041ac6a41f814755af6874f12bfdd960`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:40.698464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0b412436-fdf0-4b2a-839b-8d22eeb6f25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:40.942706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64496645-cdcd-4892-94b2-70cb67062f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:40.945133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64496645-cdcd-4892-94b2-70cb67062f37 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:40.947251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64496645-cdcd-4892-94b2-70cb67062f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:40.949154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64496645-cdcd-4892-94b2-70cb67062f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:40.950802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64496645-cdcd-4892-94b2-70cb67062f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:40.957629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64496645-cdcd-4892-94b2-70cb67062f37 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:40.962573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 897/1795] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:40.964323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b412436-fdf0-4b2a-839b-8d22eeb6f25c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:41.012894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b412436-fdf0-4b2a-839b-8d22eeb6f25c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:41.027085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 899/1796] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:40 2022] POST /identity/v3/projects => generated 383 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:41.031239 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64496645-cdcd-4892-94b2-70cb67062f37 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:41.042250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3b655-4aa6-4841-b772-8f55abf010fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:41.043886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3b655-4aa6-4841-b772-8f55abf010fc admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:41.045428 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3b655-4aa6-4841-b772-8f55abf010fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:41.045711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3b655-4aa6-4841-b772-8f55abf010fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:41.051878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3b655-4aa6-4841-b772-8f55abf010fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:41.058948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3b655-4aa6-4841-b772-8f55abf010fc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:41.143438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 898/1797] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:40 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:41.148706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3b655-4aa6-4841-b772-8f55abf010fc admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:41.170817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64496645-cdcd-4892-94b2-70cb67062f37 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:41.326749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3b655-4aa6-4841-b772-8f55abf010fc admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:41.333217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b50d73-2b47-460b-9f87-1230388c4ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:41.334181 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b50d73-2b47-460b-9f87-1230388c4ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:41.334678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b50d73-2b47-460b-9f87-1230388c4ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:41.335190 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b50d73-2b47-460b-9f87-1230388c4ae2 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:41.336082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b50d73-2b47-460b-9f87-1230388c4ae2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:41.340150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b50d73-2b47-460b-9f87-1230388c4ae2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:41.351550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:41.354466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb24749-4081-49a9-8a87-024cd6fbab7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:41.356096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb24749-4081-49a9-8a87-024cd6fbab7e admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:41.356602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb24749-4081-49a9-8a87-024cd6fbab7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:41.357587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb24749-4081-49a9-8a87-024cd6fbab7e admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:41.359066 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb24749-4081-49a9-8a87-024cd6fbab7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:41.363188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb24749-4081-49a9-8a87-024cd6fbab7e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:41.375554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 899/1798] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:41 2022] POST /identity/v3/users => generated 298 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:41.380167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b50d73-2b47-460b-9f87-1230388c4ae2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:41.394319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b50d73-2b47-460b-9f87-1230388c4ae2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:41.399847 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985eaba-3235-48ae-b2a8-1b3fd8cdc3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:41.400876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985eaba-3235-48ae-b2a8-1b3fd8cdc3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:41.401255 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985eaba-3235-48ae-b2a8-1b3fd8cdc3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:41.401809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985eaba-3235-48ae-b2a8-1b3fd8cdc3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:41.402674 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985eaba-3235-48ae-b2a8-1b3fd8cdc3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:41.404576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985eaba-3235-48ae-b2a8-1b3fd8cdc3f6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:41.419262 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 900/1799] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:23:41.419998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:41.424056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985eaba-3235-48ae-b2a8-1b3fd8cdc3f6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:41.444764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985eaba-3235-48ae-b2a8-1b3fd8cdc3f6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:41.452470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abd9ea-cc4f-45fd-a52f-fa05ee979660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:41.453781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abd9ea-cc4f-45fd-a52f-fa05ee979660 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:41.454341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abd9ea-cc4f-45fd-a52f-fa05ee979660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:41.454861 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abd9ea-cc4f-45fd-a52f-fa05ee979660 admin admin] PATH_INFO: `/v3/projects/cfae3a96d3104d06b71c3d7e62d38bad/users/a9d26d63903c4b25835a580ca295a78a/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:41.471750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 900/1800] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:41 2022] POST /identity/v3/users => generated 298 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:41.479719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abd9ea-cc4f-45fd-a52f-fa05ee979660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfae3a96d3104d06b71c3d7e62d38bad, user_id=a9d26d63903c4b25835a580ca295a78a, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:41.482951 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abd9ea-cc4f-45fd-a52f-fa05ee979660 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:41.495858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb24749-4081-49a9-8a87-024cd6fbab7e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:41.518180 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb24749-4081-49a9-8a87-024cd6fbab7e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:41.534878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 901/1801] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:41 2022] PUT /identity/v3/projects/cfae3a96d3104d06b71c3d7e62d38bad/users/a9d26d63903c4b25835a580ca295a78a/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:41.539161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abd9ea-cc4f-45fd-a52f-fa05ee979660 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:41.673339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abd9ea-cc4f-45fd-a52f-fa05ee979660 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:41.686845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d88a55-305d-4fa7-8e2d-586d03cec63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:41.690160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d88a55-305d-4fa7-8e2d-586d03cec63a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:41.690541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d88a55-305d-4fa7-8e2d-586d03cec63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:41.691556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bc4ae-bdfc-4512-a369-fec242dbe459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:41.691982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d88a55-305d-4fa7-8e2d-586d03cec63a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:41.693129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d88a55-305d-4fa7-8e2d-586d03cec63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:41.693821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bc4ae-bdfc-4512-a369-fec242dbe459 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:41.694525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bc4ae-bdfc-4512-a369-fec242dbe459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:41.695112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077bc4ae-bdfc-4512-a369-fec242dbe459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:41.696319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d88a55-305d-4fa7-8e2d-586d03cec63a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:41.697052 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077bc4ae-bdfc-4512-a369-fec242dbe459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:41.700565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077bc4ae-bdfc-4512-a369-fec242dbe459 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:41.724294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 902/1802] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:41.726413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d88a55-305d-4fa7-8e2d-586d03cec63a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:41.739759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 901/1803] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:41.740957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d88a55-305d-4fa7-8e2d-586d03cec63a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:41.745904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bc4ae-bdfc-4512-a369-fec242dbe459 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:41.749780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39d5e7-5cf9-436a-8ffc-62473308fcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:41.750782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39d5e7-5cf9-436a-8ffc-62473308fcd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:41.751166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39d5e7-5cf9-436a-8ffc-62473308fcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:41.751540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39d5e7-5cf9-436a-8ffc-62473308fcd6 admin admin] PATH_INFO: `/v3/projects/cfae3a96d3104d06b71c3d7e62d38bad/users/a9d26d63903c4b25835a580ca295a78a/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:41.754414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39d5e7-5cf9-436a-8ffc-62473308fcd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfae3a96d3104d06b71c3d7e62d38bad, user_id=a9d26d63903c4b25835a580ca295a78a, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:41.761482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39d5e7-5cf9-436a-8ffc-62473308fcd6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:41.773977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077bc4ae-bdfc-4512-a369-fec242dbe459 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:41.779356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 903/1804] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:41 2022] PUT /identity/v3/projects/cfae3a96d3104d06b71c3d7e62d38bad/users/a9d26d63903c4b25835a580ca295a78a/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:41.795269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaea07-ac47-4416-a058-7d136340be3e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:41.795701 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaea07-ac47-4416-a058-7d136340be3e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:41.796852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aaea07-ac47-4416-a058-7d136340be3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:41.864929 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-06aaea07-ac47-4416-a058-7d136340be3e None None] MFA Rules not processed for user `a9d26d63903c4b25835a580ca295a78a`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:41.867227 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-06aaea07-ac47-4416-a058-7d136340be3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:41.883317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e68f4d-1a4e-4506-a3ff-be8162c3f808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:41.884419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e68f4d-1a4e-4506-a3ff-be8162c3f808 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:41.885223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e68f4d-1a4e-4506-a3ff-be8162c3f808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:41.886535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e68f4d-1a4e-4506-a3ff-be8162c3f808 admin admin] PATH_INFO: `/v3/projects/3ecaa2bbb2fe469cbfe2b8a7ef241ac5/users/1a323adfdd394b13aefe57054b15ac6c/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:41.909069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e68f4d-1a4e-4506-a3ff-be8162c3f808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ecaa2bbb2fe469cbfe2b8a7ef241ac5, user_id=1a323adfdd394b13aefe57054b15ac6c, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:41.913552 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e68f4d-1a4e-4506-a3ff-be8162c3f808 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:41.935624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 902/1805] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:41 2022] PUT /identity/v3/projects/3ecaa2bbb2fe469cbfe2b8a7ef241ac5/users/1a323adfdd394b13aefe57054b15ac6c/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:41.941281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e68f4d-1a4e-4506-a3ff-be8162c3f808 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:42.059636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e68f4d-1a4e-4506-a3ff-be8162c3f808 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:42.073219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95365549-b94f-4ede-b297-9908f32f03bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:42.074961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95365549-b94f-4ede-b297-9908f32f03bc admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:42.075727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95365549-b94f-4ede-b297-9908f32f03bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:42.076928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95365549-b94f-4ede-b297-9908f32f03bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:42.079522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95365549-b94f-4ede-b297-9908f32f03bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:42.083741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95365549-b94f-4ede-b297-9908f32f03bc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:42.091742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 904/1806] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:42.100804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aaea07-ac47-4416-a058-7d136340be3e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:42.102632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-06aaea07-ac47-4416-a058-7d136340be3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:42.118636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 903/1807] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:23:42.124966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95365549-b94f-4ede-b297-9908f32f03bc admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:42.143344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95365549-b94f-4ede-b297-9908f32f03bc admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:42.149170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4a91da-342b-4c33-949e-f876b3b50d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:42.150222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a91da-342b-4c33-949e-f876b3b50d73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:42.150791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a91da-342b-4c33-949e-f876b3b50d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:42.151176 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4a91da-342b-4c33-949e-f876b3b50d73 admin admin] PATH_INFO: `/v3/projects/3ecaa2bbb2fe469cbfe2b8a7ef241ac5/users/1a323adfdd394b13aefe57054b15ac6c/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:42.153517 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4a91da-342b-4c33-949e-f876b3b50d73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ecaa2bbb2fe469cbfe2b8a7ef241ac5, user_id=1a323adfdd394b13aefe57054b15ac6c, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:42.157624 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4a91da-342b-4c33-949e-f876b3b50d73 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:42.161493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:42.170984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:42.179333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 904/1808] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:42 2022] PUT /identity/v3/projects/3ecaa2bbb2fe469cbfe2b8a7ef241ac5/users/1a323adfdd394b13aefe57054b15ac6c/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:42.199108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bc313-108c-4295-84ed-38a169b5c661 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:42.200362 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bc313-108c-4295-84ed-38a169b5c661 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:42.200748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bc313-108c-4295-84ed-38a169b5c661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:42.216329 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aaea07-ac47-4416-a058-7d136340be3e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:42.257998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-048bc313-108c-4295-84ed-38a169b5c661 None None] MFA Rules not processed for user `1a323adfdd394b13aefe57054b15ac6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:42.259390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-048bc313-108c-4295-84ed-38a169b5c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:42.423741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafa332-00aa-42cf-8a51-46afb38b2877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:42.424818 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafa332-00aa-42cf-8a51-46afb38b2877 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:42.425313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafa332-00aa-42cf-8a51-46afb38b2877 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:42.425786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafa332-00aa-42cf-8a51-46afb38b2877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:42.467819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 905/1809] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:23:42.491467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bc313-108c-4295-84ed-38a169b5c661 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:42.492983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-048bc313-108c-4295-84ed-38a169b5c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:42.522335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bc313-108c-4295-84ed-38a169b5c661 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:42.533713 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7baba-7674-4ad1-b7c3-948fd4e19902 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:42.535234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7baba-7674-4ad1-b7c3-948fd4e19902 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:42.535583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7baba-7674-4ad1-b7c3-948fd4e19902 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:42.536596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7baba-7674-4ad1-b7c3-948fd4e19902 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:42.565681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7baba-7674-4ad1-b7c3-948fd4e19902 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:42.570666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a7baba-7674-4ad1-b7c3-948fd4e19902 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:42.611910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 906/1810] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:23:42.661980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafa332-00aa-42cf-8a51-46afb38b2877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:42.664002 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafa332-00aa-42cf-8a51-46afb38b2877 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:42.692573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 905/1811] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 592 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:23:42.848993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7baba-7674-4ad1-b7c3-948fd4e19902 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:42.865432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7baba-7674-4ad1-b7c3-948fd4e19902 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:42.868186 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d2d5d-c4d1-40ac-8d86-896cbd6135b7 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:42.869038 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d2d5d-c4d1-40ac-8d86-896cbd6135b7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:42.869038 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d2d5d-c4d1-40ac-8d86-896cbd6135b7 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:42.871830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 906/1812] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:23:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:23:42.884091 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1103b-6a39-4f70-b25e-1182e4d28837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:42.885969 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1103b-6a39-4f70-b25e-1182e4d28837 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:42.886515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1103b-6a39-4f70-b25e-1182e4d28837 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:42.891239 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1103b-6a39-4f70-b25e-1182e4d28837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:42.898173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d2d5d-c4d1-40ac-8d86-896cbd6135b7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:42.900139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-734d2d5d-c4d1-40ac-8d86-896cbd6135b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:43.076920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1103b-6a39-4f70-b25e-1182e4d28837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:43.082464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1103b-6a39-4f70-b25e-1182e4d28837 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:43.125446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 907/1813] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:43.143792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eaed0c-54a2-42ee-a47f-e3c939af0e52 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:43.144134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eaed0c-54a2-42ee-a47f-e3c939af0e52 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:43.144419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eaed0c-54a2-42ee-a47f-e3c939af0e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:43.173291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d2d5d-c4d1-40ac-8d86-896cbd6135b7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:43.185662 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:43.189740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:43.190209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:43.190518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] PATH_INFO: `/v3/projects/2e28ffc92a704580aa12641b62d0d404` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:43.196177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-28eaed0c-54a2-42ee-a47f-e3c939af0e52 None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:43.196733 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e28ffc92a704580aa12641b62d0d404)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:43.200228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-28eaed0c-54a2-42ee-a47f-e3c939af0e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:43.201166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:43.315660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:43.355529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e28ffc92a704580aa12641b62d0d404', 'request_id': 'req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:43.376484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e28ffc92a704580aa12641b62d0d404', 'request_id': 'req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:43.388823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e28ffc92a704580aa12641b62d0d404', 'request_id': 'req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:43.407817 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] The token cache is being invalidate because project 2e28ffc92a704580aa12641b62d0d404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:43.407817 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:43.409804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:43.412216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:43.417370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 907/1814] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:42 2022] DELETE /identity/v3/projects/2e28ffc92a704580aa12641b62d0d404 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:43.423561 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:43.425238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:43.477756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 908/1815] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:23:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:43.586418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9965cc9a-f11b-4b8b-ae9a-fe05c72b5b7e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:43.603473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:43.606755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:43.607811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:43.608047 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] PATH_INFO: `/v3/users/698277dfe4ae4f53823a6a9cb41e77ad` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:43.610165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=698277dfe4ae4f53823a6a9cb41e77ad)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:43.614173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:43.835351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '698277dfe4ae4f53823a6a9cb41e77ad'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:43.879748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '698277dfe4ae4f53823a6a9cb41e77ad'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:43.984097 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '698277dfe4ae4f53823a6a9cb41e77ad'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:43.991613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '698277dfe4ae4f53823a6a9cb41e77ad'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:44.054368 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '698277dfe4ae4f53823a6a9cb41e77ad'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:44.075186 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 908/1816] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:43 2022] DELETE /identity/v3/users/698277dfe4ae4f53823a6a9cb41e77ad => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:44.439416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eaed0c-54a2-42ee-a47f-e3c939af0e52 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:44.441640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-28eaed0c-54a2-42ee-a47f-e3c939af0e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:44.461431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eaed0c-54a2-42ee-a47f-e3c939af0e52 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:44.614366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056db9e-6342-45e5-a400-247a794e2268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:44.615773 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056db9e-6342-45e5-a400-247a794e2268 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:44.616230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056db9e-6342-45e5-a400-247a794e2268 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:44.616646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6056db9e-6342-45e5-a400-247a794e2268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:44.619009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6056db9e-6342-45e5-a400-247a794e2268 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:44.641483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056db9e-6342-45e5-a400-247a794e2268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:44.644593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6056db9e-6342-45e5-a400-247a794e2268 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:44.679206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 909/1817] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:23:45.305240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:45.307567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:45.517187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:45.584194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8dafd-2822-4558-b487-275b108d0183 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:45.595152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fa973-6078-4ce4-8cc9-2388870f1b8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:45.596210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fa973-6078-4ce4-8cc9-2388870f1b8d service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:45.598064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fa973-6078-4ce4-8cc9-2388870f1b8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:45.598688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fa973-6078-4ce4-8cc9-2388870f1b8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:45.603754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-170fa973-6078-4ce4-8cc9-2388870f1b8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:45.817812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fa973-6078-4ce4-8cc9-2388870f1b8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:45.819918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fa973-6078-4ce4-8cc9-2388870f1b8d service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:45.832266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 909/1818] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:46.226997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056db9e-6342-45e5-a400-247a794e2268 service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:46.231121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6056db9e-6342-45e5-a400-247a794e2268 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:46.394336 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6056db9e-6342-45e5-a400-247a794e2268 service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:46.412852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a6efa-12f6-4498-8fac-addc9a3b5da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:46.413997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a6efa-12f6-4498-8fac-addc9a3b5da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:46.414492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a6efa-12f6-4498-8fac-addc9a3b5da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:46.415157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a6efa-12f6-4498-8fac-addc9a3b5da5 admin admin] PATH_INFO: `/v3/projects/eebcb4626f7c4c4aaf70c1f8dfac2f4c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:46.417300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a6efa-12f6-4498-8fac-addc9a3b5da5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eebcb4626f7c4c4aaf70c1f8dfac2f4c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:46.420903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773a6efa-12f6-4498-8fac-addc9a3b5da5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:46.583572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-773a6efa-12f6-4498-8fac-addc9a3b5da5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eebcb4626f7c4c4aaf70c1f8dfac2f4c', 'request_id': 'req-773a6efa-12f6-4498-8fac-addc9a3b5da5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:46.605748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-773a6efa-12f6-4498-8fac-addc9a3b5da5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eebcb4626f7c4c4aaf70c1f8dfac2f4c', 'request_id': 'req-773a6efa-12f6-4498-8fac-addc9a3b5da5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:46.621552 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-773a6efa-12f6-4498-8fac-addc9a3b5da5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eebcb4626f7c4c4aaf70c1f8dfac2f4c', 'request_id': 'req-773a6efa-12f6-4498-8fac-addc9a3b5da5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:46.654208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-773a6efa-12f6-4498-8fac-addc9a3b5da5 admin admin] The token cache is being invalidate because project eebcb4626f7c4c4aaf70c1f8dfac2f4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:46.654941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-773a6efa-12f6-4498-8fac-addc9a3b5da5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:46.656758 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-773a6efa-12f6-4498-8fac-addc9a3b5da5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:46.668316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 910/1819] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:46 2022] DELETE /identity/v3/projects/eebcb4626f7c4c4aaf70c1f8dfac2f4c => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:46.668586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba7abf-4d0b-45b6-978d-9aee03d57e26 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:46.668586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba7abf-4d0b-45b6-978d-9aee03d57e26 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:46.668586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba7abf-4d0b-45b6-978d-9aee03d57e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:46.735199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7bba7abf-4d0b-45b6-978d-9aee03d57e26 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:46.737554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7bba7abf-4d0b-45b6-978d-9aee03d57e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:46.991245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 910/1820] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:47.025662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0aa67-db93-4d4d-9524-b5e73b91d66a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:47.026317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0aa67-db93-4d4d-9524-b5e73b91d66a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:47.027600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0aa67-db93-4d4d-9524-b5e73b91d66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:47.102087 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bce78-f5fd-4d16-8214-2fefb0070939 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:47.102958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bce78-f5fd-4d16-8214-2fefb0070939 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:47.104202 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99bce78-f5fd-4d16-8214-2fefb0070939 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:47.109706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 911/1821] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:23:47 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:23:47.134929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-12f0aa67-db93-4d4d-9524-b5e73b91d66a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:47.137321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-12f0aa67-db93-4d4d-9524-b5e73b91d66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:47.163047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 911/1822] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:47.166963 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bce78-f5fd-4d16-8214-2fefb0070939 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:47.169584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4299d0-bd36-49d9-977f-ac49a59d8234 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:47.170453 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4299d0-bd36-49d9-977f-ac49a59d8234 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:47.171104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4299d0-bd36-49d9-977f-ac49a59d8234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:47.186646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99bce78-f5fd-4d16-8214-2fefb0070939 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:47.198161 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:47.198458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244752cf-fa0d-41f9-b838-b3c284c4a21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:47.199333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244752cf-fa0d-41f9-b838-b3c284c4a21b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:47.199333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244752cf-fa0d-41f9-b838-b3c284c4a21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:47.199658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244752cf-fa0d-41f9-b838-b3c284c4a21b admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:47.200356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244752cf-fa0d-41f9-b838-b3c284c4a21b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:23:47.200646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244752cf-fa0d-41f9-b838-b3c284c4a21b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:47.202575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244752cf-fa0d-41f9-b838-b3c284c4a21b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:47.245943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1d4299d0-bd36-49d9-977f-ac49a59d8234 None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:47.248081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1d4299d0-bd36-49d9-977f-ac49a59d8234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:47.268540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 912/1823] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:23:47.270285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244752cf-fa0d-41f9-b838-b3c284c4a21b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:47.277284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-244752cf-fa0d-41f9-b838-b3c284c4a21b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:47.343378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244752cf-fa0d-41f9-b838-b3c284c4a21b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:47.365990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:47.367182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:47.367855 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:47.368354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] PATH_INFO: `/v3/users/81c831e716d544e885d951de805edccc` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:47.376243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c831e716d544e885d951de805edccc)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:47.380402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:47.551008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c831e716d544e885d951de805edccc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:47.564938 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c831e716d544e885d951de805edccc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:47.597959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c831e716d544e885d951de805edccc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:47.601309 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c831e716d544e885d951de805edccc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:47.619077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 912/1824] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:23:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:23:47.620425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4299d0-bd36-49d9-977f-ac49a59d8234 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:47.628985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1d4299d0-bd36-49d9-977f-ac49a59d8234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:47.632323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:47.639306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c831e716d544e885d951de805edccc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:47.664636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 913/1825] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:47 2022] DELETE /identity/v3/users/81c831e716d544e885d951de805edccc => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:47.668003 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:47.674884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:47.695695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4299d0-bd36-49d9-977f-ac49a59d8234 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:47.852943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d61810-9946-4201-ac12-94ada7fa6c98 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:47.854941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0c03b-7b87-4433-a2c6-81ac2be727bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:47.856216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0c03b-7b87-4433-a2c6-81ac2be727bf admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:47.856501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0c03b-7b87-4433-a2c6-81ac2be727bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:47.856779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0c03b-7b87-4433-a2c6-81ac2be727bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:47.857561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0c03b-7b87-4433-a2c6-81ac2be727bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:47.859913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83212147-bfbb-4f6c-b7d0-304c77bc7fba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:47.860162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0c03b-7b87-4433-a2c6-81ac2be727bf admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:47.860964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83212147-bfbb-4f6c-b7d0-304c77bc7fba service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:47.861508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83212147-bfbb-4f6c-b7d0-304c77bc7fba service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:47.861923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83212147-bfbb-4f6c-b7d0-304c77bc7fba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:47.865961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-83212147-bfbb-4f6c-b7d0-304c77bc7fba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:47.904939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 913/1826] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:47 2022] POST /identity/v3/projects => generated 355 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:47.907305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d2d51-f2bf-4cd8-afe8-8358f0a8f441 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:47.907600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d2d51-f2bf-4cd8-afe8-8358f0a8f441 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:47.907985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d2d51-f2bf-4cd8-afe8-8358f0a8f441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:47.954741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7e4d2d51-f2bf-4cd8-afe8-8358f0a8f441 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:47.956023 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7e4d2d51-f2bf-4cd8-afe8-8358f0a8f441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:48.029267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83212147-bfbb-4f6c-b7d0-304c77bc7fba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:48.033597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83212147-bfbb-4f6c-b7d0-304c77bc7fba service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:48.051127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 914/1827] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:48.052648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83212147-bfbb-4f6c-b7d0-304c77bc7fba service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:48.067204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83212147-bfbb-4f6c-b7d0-304c77bc7fba service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:48.093526 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 914/1828] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:48.123557 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d2d51-f2bf-4cd8-afe8-8358f0a8f441 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:48.125428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7e4d2d51-f2bf-4cd8-afe8-8358f0a8f441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:48.199240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff32507-64da-43a6-8588-2ff95f15279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:48.200136 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff32507-64da-43a6-8588-2ff95f15279c admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:48.200748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff32507-64da-43a6-8588-2ff95f15279c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:48.201395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff32507-64da-43a6-8588-2ff95f15279c admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:48.202316 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff32507-64da-43a6-8588-2ff95f15279c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:48.205558 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff32507-64da-43a6-8588-2ff95f15279c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:48.221247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:48.273299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 915/1829] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:48 2022] POST /identity/v3/users => generated 278 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:48.273299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff32507-64da-43a6-8588-2ff95f15279c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:48.279380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dff32507-64da-43a6-8588-2ff95f15279c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:48.309933 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d2d51-f2bf-4cd8-afe8-8358f0a8f441 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:48.317400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e8bbb-b739-41e7-a6ee-11a7c2dcce55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:48.319320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e8bbb-b739-41e7-a6ee-11a7c2dcce55 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:48.319883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e8bbb-b739-41e7-a6ee-11a7c2dcce55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:48.320448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e8bbb-b739-41e7-a6ee-11a7c2dcce55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:48.337329 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e8bbb-b739-41e7-a6ee-11a7c2dcce55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:48.341052 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e8bbb-b739-41e7-a6ee-11a7c2dcce55 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:48.353034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:48.376363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 915/1830] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:23:48.380545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e8bbb-b739-41e7-a6ee-11a7c2dcce55 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:48.416983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff32507-64da-43a6-8588-2ff95f15279c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:48.422796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e8bbb-b739-41e7-a6ee-11a7c2dcce55 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:48.425187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45e1e5-2408-487c-b9bd-957504099c24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:48.426072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45e1e5-2408-487c-b9bd-957504099c24 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:48.426438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45e1e5-2408-487c-b9bd-957504099c24 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:48.426958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45e1e5-2408-487c-b9bd-957504099c24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:48.438369 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45e1e5-2408-487c-b9bd-957504099c24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:48.440988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45e1e5-2408-487c-b9bd-957504099c24 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:48.446252 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd861211-9c50-43d2-9449-8785315657c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:48.447398 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd861211-9c50-43d2-9449-8785315657c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:48.448372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd861211-9c50-43d2-9449-8785315657c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:48.448372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd861211-9c50-43d2-9449-8785315657c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:48.449440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd861211-9c50-43d2-9449-8785315657c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:48.458758 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd861211-9c50-43d2-9449-8785315657c9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:48.479306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 916/1831] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:48.503986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 916/1832] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:48.506939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45e1e5-2408-487c-b9bd-957504099c24 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:48.533194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45e1e5-2408-487c-b9bd-957504099c24 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:48.551719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a019c994-02ac-4f26-96e6-df9c0ae83c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:48.552981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019c994-02ac-4f26-96e6-df9c0ae83c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:48.553544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019c994-02ac-4f26-96e6-df9c0ae83c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:48.554640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a019c994-02ac-4f26-96e6-df9c0ae83c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:48.556866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019c994-02ac-4f26-96e6-df9c0ae83c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:48.563460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a019c994-02ac-4f26-96e6-df9c0ae83c63 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:48.593998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 917/1833] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:48.599788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd861211-9c50-43d2-9449-8785315657c9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:48.626481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd861211-9c50-43d2-9449-8785315657c9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:48.647724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2e302-b823-4eb9-8cc0-47654decf90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:48.648866 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2e302-b823-4eb9-8cc0-47654decf90c admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:48.649352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2e302-b823-4eb9-8cc0-47654decf90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:48.649841 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2e302-b823-4eb9-8cc0-47654decf90c admin admin] PATH_INFO: `/v3/projects/509e582db536430191010d80d2d1f10b/users/6e83ee8ae7264504899875d4a8303a83/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:48.721469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2e302-b823-4eb9-8cc0-47654decf90c admin admin] RBAC: Authorizing `identity:create_grant(project_id=509e582db536430191010d80d2d1f10b, user_id=6e83ee8ae7264504899875d4a8303a83, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:48.730306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2e302-b823-4eb9-8cc0-47654decf90c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:48.789849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 917/1834] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:48 2022] PUT /identity/v3/projects/509e582db536430191010d80d2d1f10b/users/6e83ee8ae7264504899875d4a8303a83/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:48.791301 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151493-7ca1-4845-b723-19f0c90dbbe3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:48.791918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151493-7ca1-4845-b723-19f0c90dbbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:48.792338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd151493-7ca1-4845-b723-19f0c90dbbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:48.860528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fd151493-7ca1-4845-b723-19f0c90dbbe3 None None] MFA Rules not processed for user `6e83ee8ae7264504899875d4a8303a83`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:48.861879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fd151493-7ca1-4845-b723-19f0c90dbbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:49.067944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 918/1835] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:48 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:49.070121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2e302-b823-4eb9-8cc0-47654decf90c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:49.251274 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2e302-b823-4eb9-8cc0-47654decf90c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:49.260194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c169aa-ed82-45f9-a867-5f1955c14d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:49.260837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c169aa-ed82-45f9-a867-5f1955c14d2e admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:49.261112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c169aa-ed82-45f9-a867-5f1955c14d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:49.265632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c169aa-ed82-45f9-a867-5f1955c14d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:49.266514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c169aa-ed82-45f9-a867-5f1955c14d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:49.269120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c169aa-ed82-45f9-a867-5f1955c14d2e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:49.317425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 918/1836] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:49 2022] POST /identity/v3/projects => generated 357 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:49.319009 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd151493-7ca1-4845-b723-19f0c90dbbe3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:49.353936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd151493-7ca1-4845-b723-19f0c90dbbe3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:49.538758 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2519fa7-c20a-42d3-bae8-ed227ebb6424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:49.539846 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519fa7-c20a-42d3-bae8-ed227ebb6424 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:49.540319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519fa7-c20a-42d3-bae8-ed227ebb6424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:49.540825 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2519fa7-c20a-42d3-bae8-ed227ebb6424 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:49.542030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2519fa7-c20a-42d3-bae8-ed227ebb6424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:49.545980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2519fa7-c20a-42d3-bae8-ed227ebb6424 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:49.597091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 919/1837] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:49 2022] POST /identity/v3/users => generated 285 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:49.603345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c169aa-ed82-45f9-a867-5f1955c14d2e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:49.640373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c169aa-ed82-45f9-a867-5f1955c14d2e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:49.684165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f35f4-9d61-409c-b30c-6032fa571ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:49.684908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f35f4-9d61-409c-b30c-6032fa571ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:49.685484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f35f4-9d61-409c-b30c-6032fa571ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:49.691378 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f35f4-9d61-409c-b30c-6032fa571ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:49.691378 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f35f4-9d61-409c-b30c-6032fa571ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:49.691378 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2f35f4-9d61-409c-b30c-6032fa571ef6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:49.732160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 919/1838] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:49.734553 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2519fa7-c20a-42d3-bae8-ed227ebb6424 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:49.767925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2519fa7-c20a-42d3-bae8-ed227ebb6424 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:49.776632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda282f1-b04c-49f5-9107-ca200c4b0518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:49.777775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda282f1-b04c-49f5-9107-ca200c4b0518 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:49.778288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda282f1-b04c-49f5-9107-ca200c4b0518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:49.778763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda282f1-b04c-49f5-9107-ca200c4b0518 admin admin] PATH_INFO: `/v3/projects/58161c0c02b243fc98a9b33f269f1b64/users/d0ecc427cdf149cdb6d6b5ccf4fec156/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:49.812554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda282f1-b04c-49f5-9107-ca200c4b0518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58161c0c02b243fc98a9b33f269f1b64, user_id=d0ecc427cdf149cdb6d6b5ccf4fec156, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:49.817394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda282f1-b04c-49f5-9107-ca200c4b0518 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:49.862764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 920/1839] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:49 2022] PUT /identity/v3/projects/58161c0c02b243fc98a9b33f269f1b64/users/d0ecc427cdf149cdb6d6b5ccf4fec156/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:49.865411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f35f4-9d61-409c-b30c-6032fa571ef6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:49.896161 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2f35f4-9d61-409c-b30c-6032fa571ef6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:50.059472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862cc99-2f75-4288-a5c9-3c4261bfaf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:50.060381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862cc99-2f75-4288-a5c9-3c4261bfaf28 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:50.060809 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862cc99-2f75-4288-a5c9-3c4261bfaf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:50.061492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862cc99-2f75-4288-a5c9-3c4261bfaf28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:50.062482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862cc99-2f75-4288-a5c9-3c4261bfaf28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:50.066298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862cc99-2f75-4288-a5c9-3c4261bfaf28 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:50.084370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:50.096232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 920/1840] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:50.101517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda282f1-b04c-49f5-9107-ca200c4b0518 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:50.115752 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:50.123833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda282f1-b04c-49f5-9107-ca200c4b0518 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:50.133654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433643d-9074-443f-9591-3007782c3bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:50.134599 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433643d-9074-443f-9591-3007782c3bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:50.134980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433643d-9074-443f-9591-3007782c3bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:50.135314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f433643d-9074-443f-9591-3007782c3bd5 admin admin] PATH_INFO: `/v3/projects/58161c0c02b243fc98a9b33f269f1b64/users/d0ecc427cdf149cdb6d6b5ccf4fec156/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:50.137586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433643d-9074-443f-9591-3007782c3bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58161c0c02b243fc98a9b33f269f1b64, user_id=d0ecc427cdf149cdb6d6b5ccf4fec156, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:50.140595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f433643d-9074-443f-9591-3007782c3bd5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:50.153548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 921/1841] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:50 2022] PUT /identity/v3/projects/58161c0c02b243fc98a9b33f269f1b64/users/d0ecc427cdf149cdb6d6b5ccf4fec156/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:50.165040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d5e02-1f2a-4048-87dd-02284761e777 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:50.166333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d5e02-1f2a-4048-87dd-02284761e777 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:50.166712 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d5e02-1f2a-4048-87dd-02284761e777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:50.213956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f77d5e02-1f2a-4048-87dd-02284761e777 None None] MFA Rules not processed for user `d0ecc427cdf149cdb6d6b5ccf4fec156`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:50.214824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f77d5e02-1f2a-4048-87dd-02284761e777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:50.218297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433643d-9074-443f-9591-3007782c3bd5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:50.222943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f433643d-9074-443f-9591-3007782c3bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:50.395156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f433643d-9074-443f-9591-3007782c3bd5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:50.410831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:50.430739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:50.431044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:50.431996 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] PATH_INFO: `/v3/projects/dffcc69d970c4c6188311cb104fd9964` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:50.439760 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dffcc69d970c4c6188311cb104fd9964)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:50.441757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:50.445838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 921/1842] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:50 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:50.463345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d5e02-1f2a-4048-87dd-02284761e777 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:50.465547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f77d5e02-1f2a-4048-87dd-02284761e777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:50.559920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dffcc69d970c4c6188311cb104fd9964', 'request_id': 'req-f8715efd-42b7-4a18-b4e5-c196f04375e4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:50.576285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dffcc69d970c4c6188311cb104fd9964', 'request_id': 'req-f8715efd-42b7-4a18-b4e5-c196f04375e4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:50.605305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dffcc69d970c4c6188311cb104fd9964', 'request_id': 'req-f8715efd-42b7-4a18-b4e5-c196f04375e4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:50.633598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d5e02-1f2a-4048-87dd-02284761e777 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:50.642107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] The token cache is being invalidate because project dffcc69d970c4c6188311cb104fd9964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:50.642107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:50.645065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:50.650761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 922/1843] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:50 2022] DELETE /identity/v3/projects/dffcc69d970c4c6188311cb104fd9964 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:50.656934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:50.659832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:50.768856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23fcde-8979-4b2d-91ed-8f227539a78c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:50.769501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23fcde-8979-4b2d-91ed-8f227539a78c service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:50.769782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23fcde-8979-4b2d-91ed-8f227539a78c service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:50.770090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23fcde-8979-4b2d-91ed-8f227539a78c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:50.772736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5f23fcde-8979-4b2d-91ed-8f227539a78c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:50.821268 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8715efd-42b7-4a18-b4e5-c196f04375e4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:50.834675 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:50.835494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:50.835778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:50.836053 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] PATH_INFO: `/v3/users/1a323adfdd394b13aefe57054b15ac6c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:50.838923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a323adfdd394b13aefe57054b15ac6c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:50.841459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:50.897216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:50.901294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:50.946655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23fcde-8979-4b2d-91ed-8f227539a78c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:50.952142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f23fcde-8979-4b2d-91ed-8f227539a78c service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:50.961665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a323adfdd394b13aefe57054b15ac6c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:50.992784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a323adfdd394b13aefe57054b15ac6c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:50.996023 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 922/1844] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:50 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:23:51.050901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a323adfdd394b13aefe57054b15ac6c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:51.053594 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a323adfdd394b13aefe57054b15ac6c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:51.091250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a323adfdd394b13aefe57054b15ac6c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:51.107339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 923/1845] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:50 2022] DELETE /identity/v3/users/1a323adfdd394b13aefe57054b15ac6c => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:51.176366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23fcde-8979-4b2d-91ed-8f227539a78c service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:51.177948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5f23fcde-8979-4b2d-91ed-8f227539a78c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:51.200545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f23fcde-8979-4b2d-91ed-8f227539a78c service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:51.349587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f513237-35ff-41e7-aad8-76d0e8668899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:51.350265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f513237-35ff-41e7-aad8-76d0e8668899 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:51.350725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f513237-35ff-41e7-aad8-76d0e8668899 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:51.350883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f513237-35ff-41e7-aad8-76d0e8668899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:51.357890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9f513237-35ff-41e7-aad8-76d0e8668899 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:51.552731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f513237-35ff-41e7-aad8-76d0e8668899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:51.554309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f513237-35ff-41e7-aad8-76d0e8668899 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:51.577262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 923/1846] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:51 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:23:52.111612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:52.113216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:52.254779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc891e-870f-4f63-b139-79e46e5fb198 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:52.263018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a177a-b647-4b0c-a849-e0f9ef7c263d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:52.264523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a177a-b647-4b0c-a849-e0f9ef7c263d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:52.265408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a177a-b647-4b0c-a849-e0f9ef7c263d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:52.266466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a177a-b647-4b0c-a849-e0f9ef7c263d admin admin] PATH_INFO: `/v3/projects/3ecaa2bbb2fe469cbfe2b8a7ef241ac5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:52.269709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a177a-b647-4b0c-a849-e0f9ef7c263d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ecaa2bbb2fe469cbfe2b8a7ef241ac5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:52.274026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a177a-b647-4b0c-a849-e0f9ef7c263d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:52.391725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-252a177a-b647-4b0c-a849-e0f9ef7c263d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ecaa2bbb2fe469cbfe2b8a7ef241ac5', 'request_id': 'req-252a177a-b647-4b0c-a849-e0f9ef7c263d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:52.410237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-252a177a-b647-4b0c-a849-e0f9ef7c263d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ecaa2bbb2fe469cbfe2b8a7ef241ac5', 'request_id': 'req-252a177a-b647-4b0c-a849-e0f9ef7c263d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:52.440287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-252a177a-b647-4b0c-a849-e0f9ef7c263d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ecaa2bbb2fe469cbfe2b8a7ef241ac5', 'request_id': 'req-252a177a-b647-4b0c-a849-e0f9ef7c263d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:52.498317 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-252a177a-b647-4b0c-a849-e0f9ef7c263d admin admin] The token cache is being invalidate because project 3ecaa2bbb2fe469cbfe2b8a7ef241ac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:52.500753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-252a177a-b647-4b0c-a849-e0f9ef7c263d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:52.502792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-252a177a-b647-4b0c-a849-e0f9ef7c263d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:52.510214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 924/1847] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:52 2022] DELETE /identity/v3/projects/3ecaa2bbb2fe469cbfe2b8a7ef241ac5 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:52.512731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04c75d-18d0-440b-807e-0261ebfa3cdf None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:52.513366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04c75d-18d0-440b-807e-0261ebfa3cdf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:52.513862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04c75d-18d0-440b-807e-0261ebfa3cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:52.583481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9e04c75d-18d0-440b-807e-0261ebfa3cdf None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:52.591249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9e04c75d-18d0-440b-807e-0261ebfa3cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:52.820094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 924/1848] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:52.821521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05185d2d-2bfe-49a2-bf9b-95b0c5257dcf None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:52.821850 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05185d2d-2bfe-49a2-bf9b-95b0c5257dcf None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:52.822155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05185d2d-2bfe-49a2-bf9b-95b0c5257dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:52.867615 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-05185d2d-2bfe-49a2-bf9b-95b0c5257dcf None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:52.869007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-05185d2d-2bfe-49a2-bf9b-95b0c5257dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:52.892440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 925/1849] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:52.902104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04c75d-18d0-440b-807e-0261ebfa3cdf None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:52.920133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04c75d-18d0-440b-807e-0261ebfa3cdf None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:52.937674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d407f91-df75-47ec-beea-2274aa53f51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:52.939347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d407f91-df75-47ec-beea-2274aa53f51a admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:52.940096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d407f91-df75-47ec-beea-2274aa53f51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:52.940635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d407f91-df75-47ec-beea-2274aa53f51a admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:52.942547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d407f91-df75-47ec-beea-2274aa53f51a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:23:52.945962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d407f91-df75-47ec-beea-2274aa53f51a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:52.951690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d407f91-df75-47ec-beea-2274aa53f51a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:52.969859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 925/1850] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:23:52.974309 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05185d2d-2bfe-49a2-bf9b-95b0c5257dcf None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:52.987535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05185d2d-2bfe-49a2-bf9b-95b0c5257dcf None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:53.002764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc8b6f-5591-4b6b-b986-ec3df2dae6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:53.003406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc8b6f-5591-4b6b-b986-ec3df2dae6aa admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:53.010354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc8b6f-5591-4b6b-b986-ec3df2dae6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:53.010354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc8b6f-5591-4b6b-b986-ec3df2dae6aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:53.010354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc8b6f-5591-4b6b-b986-ec3df2dae6aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:53.010354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc8b6f-5591-4b6b-b986-ec3df2dae6aa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:53.077581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d407f91-df75-47ec-beea-2274aa53f51a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:53.082229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4d407f91-df75-47ec-beea-2274aa53f51a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:53.093287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 926/1851] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:52 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:53.098464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc8b6f-5591-4b6b-b986-ec3df2dae6aa admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:53.108403 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:53.113472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d407f91-df75-47ec-beea-2274aa53f51a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:53.175326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:53.266508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4bb8e3-2b47-4795-abdc-250adf0be0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:53.267229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bb8e3-2b47-4795-abdc-250adf0be0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:53.267509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bb8e3-2b47-4795-abdc-250adf0be0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:53.268887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4bb8e3-2b47-4795-abdc-250adf0be0e3 admin admin] PATH_INFO: `/v3/users/17b3a27f3e364de0a30e9dd046ae8b4d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:53.270634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4bb8e3-2b47-4795-abdc-250adf0be0e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17b3a27f3e364de0a30e9dd046ae8b4d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:53.275899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4bb8e3-2b47-4795-abdc-250adf0be0e3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:53.283690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc8b6f-5591-4b6b-b986-ec3df2dae6aa admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:53.303509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da973d-7423-457c-95b9-b22ff85a230a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:53.307859 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da973d-7423-457c-95b9-b22ff85a230a admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:53.309555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da973d-7423-457c-95b9-b22ff85a230a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:53.309865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da973d-7423-457c-95b9-b22ff85a230a admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:53.312767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da973d-7423-457c-95b9-b22ff85a230a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:53.317932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da973d-7423-457c-95b9-b22ff85a230a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:53.390778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 927/1852] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:53 2022] POST /identity/v3/users => generated 291 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:53.395446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da973d-7423-457c-95b9-b22ff85a230a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:53.418555 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9a4bb8e3-2b47-4795-abdc-250adf0be0e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17b3a27f3e364de0a30e9dd046ae8b4d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:53.436443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9a4bb8e3-2b47-4795-abdc-250adf0be0e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17b3a27f3e364de0a30e9dd046ae8b4d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:53.440009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9a4bb8e3-2b47-4795-abdc-250adf0be0e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17b3a27f3e364de0a30e9dd046ae8b4d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:53.442624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da973d-7423-457c-95b9-b22ff85a230a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:53.456530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a7df9-e849-41a3-a90c-73ca02c26492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:53.457010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9a4bb8e3-2b47-4795-abdc-250adf0be0e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17b3a27f3e364de0a30e9dd046ae8b4d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:53.457600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a7df9-e849-41a3-a90c-73ca02c26492 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:53.458118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a7df9-e849-41a3-a90c-73ca02c26492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:53.458614 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a7df9-e849-41a3-a90c-73ca02c26492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:53.459872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a7df9-e849-41a3-a90c-73ca02c26492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:53.483323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9a4bb8e3-2b47-4795-abdc-250adf0be0e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17b3a27f3e364de0a30e9dd046ae8b4d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:53.496053 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a7df9-e849-41a3-a90c-73ca02c26492 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:53.508919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 926/1853] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:53 2022] DELETE /identity/v3/users/17b3a27f3e364de0a30e9dd046ae8b4d => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:53.513842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e236-d1f4-45c6-bd2f-ccf7348beeef None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:53.514226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e236-d1f4-45c6-bd2f-ccf7348beeef None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:53.514445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e236-d1f4-45c6-bd2f-ccf7348beeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:53.545628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 928/1854] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:23:53.550975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a7df9-e849-41a3-a90c-73ca02c26492 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:53.557209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-152a7df9-e849-41a3-a90c-73ca02c26492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:53.588571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a7df9-e849-41a3-a90c-73ca02c26492 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:53.619833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b4d0e236-d1f4-45c6-bd2f-ccf7348beeef None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:53.622177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b4d0e236-d1f4-45c6-bd2f-ccf7348beeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:53.762210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f61770-c9e7-4d29-bb32-36f72e35df38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:53.762974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f61770-c9e7-4d29-bb32-36f72e35df38 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:53.763435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f61770-c9e7-4d29-bb32-36f72e35df38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:53.763715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f61770-c9e7-4d29-bb32-36f72e35df38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:53.766188 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f61770-c9e7-4d29-bb32-36f72e35df38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:53.768141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f61770-c9e7-4d29-bb32-36f72e35df38 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:53.813043 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 929/1855] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:23:53.814556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f61770-c9e7-4d29-bb32-36f72e35df38 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:53.832736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f61770-c9e7-4d29-bb32-36f72e35df38 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:53.840479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f5fba7-e236-4a5e-9ae3-2b11f7b11960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:53.841810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5fba7-e236-4a5e-9ae3-2b11f7b11960 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:53.842289 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5fba7-e236-4a5e-9ae3-2b11f7b11960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:53.843765 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5fba7-e236-4a5e-9ae3-2b11f7b11960 admin admin] PATH_INFO: `/v3/projects/8a89eb0302d84788baa565fe287e4a6d/users/e26c3d53c47140c2b649477f8c567828/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:53.851678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 927/1856] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:53.888505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0e236-d1f4-45c6-bd2f-ccf7348beeef None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:53.891349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b4d0e236-d1f4-45c6-bd2f-ccf7348beeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:53.909971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5fba7-e236-4a5e-9ae3-2b11f7b11960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a89eb0302d84788baa565fe287e4a6d, user_id=e26c3d53c47140c2b649477f8c567828, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:53.915294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f5fba7-e236-4a5e-9ae3-2b11f7b11960 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:53.975014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 930/1857] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:53 2022] PUT /identity/v3/projects/8a89eb0302d84788baa565fe287e4a6d/users/e26c3d53c47140c2b649477f8c567828/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:54.005852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59613595-3614-409b-9af3-84e77f51fa3c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:54.013149 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59613595-3614-409b-9af3-84e77f51fa3c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:54.013149 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59613595-3614-409b-9af3-84e77f51fa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:54.078788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0e236-d1f4-45c6-bd2f-ccf7348beeef None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:54.090089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-59613595-3614-409b-9af3-84e77f51fa3c None None] MFA Rules not processed for user `e26c3d53c47140c2b649477f8c567828`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:54.091483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-59613595-3614-409b-9af3-84e77f51fa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:54.092612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:54.176742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479af7b-b2cb-4b90-ad01-044156f2f695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:54.177815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479af7b-b2cb-4b90-ad01-044156f2f695 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:54.178361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479af7b-b2cb-4b90-ad01-044156f2f695 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:54.179860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479af7b-b2cb-4b90-ad01-044156f2f695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:54.213282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:54.265807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479af7b-b2cb-4b90-ad01-044156f2f695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:54.267640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479af7b-b2cb-4b90-ad01-044156f2f695 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:54.287477 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 928/1858] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:54.293112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 931/1859] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:54.313016 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479af7b-b2cb-4b90-ad01-044156f2f695 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:54.314848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9479af7b-b2cb-4b90-ad01-044156f2f695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:54.393470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479af7b-b2cb-4b90-ad01-044156f2f695 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:54.400365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d59a0-0b08-423b-8718-5d55e2a55e41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:54.401170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d59a0-0b08-423b-8718-5d55e2a55e41 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:54.401506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d59a0-0b08-423b-8718-5d55e2a55e41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:54.401768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d59a0-0b08-423b-8718-5d55e2a55e41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:54.414233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d59a0-0b08-423b-8718-5d55e2a55e41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:54.417497 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d59a0-0b08-423b-8718-5d55e2a55e41 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:54.437094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 929/1860] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:23:54.469692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59613595-3614-409b-9af3-84e77f51fa3c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:54.488959 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59613595-3614-409b-9af3-84e77f51fa3c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:54.515174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:54.516116 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:54.516602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:54.517037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] PATH_INFO: `/v3/users/e26c3d53c47140c2b649477f8c567828` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:54.519243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e26c3d53c47140c2b649477f8c567828)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:54.527364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:54.659466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e26c3d53c47140c2b649477f8c567828'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:54.678079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e26c3d53c47140c2b649477f8c567828'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:54.704070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e26c3d53c47140c2b649477f8c567828'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:54.709829 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e26c3d53c47140c2b649477f8c567828'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:54.734583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e26c3d53c47140c2b649477f8c567828'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:54.753136 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 932/1861] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:54 2022] DELETE /identity/v3/users/e26c3d53c47140c2b649477f8c567828 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:54.760309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6b510-af5d-4619-854c-88f49aa73ff4 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:54.761401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6b510-af5d-4619-854c-88f49aa73ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:54.761865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6b510-af5d-4619-854c-88f49aa73ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:54.826843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-41d6b510-af5d-4619-854c-88f49aa73ff4 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:54.829398 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-41d6b510-af5d-4619-854c-88f49aa73ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:54.925465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:54.927235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:55.005057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 930/1862] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:55.019254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6b510-af5d-4619-854c-88f49aa73ff4 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:55.020891 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-41d6b510-af5d-4619-854c-88f49aa73ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:55.031747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:55.156893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c38e1-d22a-4e63-987f-832ccdc8b8f6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:55.178963 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:55.179642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:55.179922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:55.180201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] PATH_INFO: `/v3/projects/5d14034d38f4450090b2c529bd45b4b9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:55.183053 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d14034d38f4450090b2c529bd45b4b9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:55.188453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:55.265297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d6b510-af5d-4619-854c-88f49aa73ff4 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:55.273414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e0cf7-fd84-478c-8474-856acabc6753 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:55.275082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e0cf7-fd84-478c-8474-856acabc6753 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:55.275814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e0cf7-fd84-478c-8474-856acabc6753 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:55.276447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e0cf7-fd84-478c-8474-856acabc6753 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:55.300502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e0cf7-fd84-478c-8474-856acabc6753 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:55.303796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e0cf7-fd84-478c-8474-856acabc6753 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:55.328262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 931/1863] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:23:55.358272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d14034d38f4450090b2c529bd45b4b9', 'request_id': 'req-b343da88-a972-495c-9bdc-7b2bfe069927'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:55.381629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d14034d38f4450090b2c529bd45b4b9', 'request_id': 'req-b343da88-a972-495c-9bdc-7b2bfe069927'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:55.387986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:55.414587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d14034d38f4450090b2c529bd45b4b9', 'request_id': 'req-b343da88-a972-495c-9bdc-7b2bfe069927'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:55.473968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] The token cache is being invalidate because project 5d14034d38f4450090b2c529bd45b4b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:55.474655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:55.479552 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:55.484455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 933/1864] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:54 2022] DELETE /identity/v3/projects/5d14034d38f4450090b2c529bd45b4b9 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:55.496397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e0cf7-fd84-478c-8474-856acabc6753 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:55.541672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e0cf7-fd84-478c-8474-856acabc6753 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:55.845840 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34560a6-0d97-4080-bd6e-d73626a56c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:55.847322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34560a6-0d97-4080-bd6e-d73626a56c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:55.847572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34560a6-0d97-4080-bd6e-d73626a56c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:55.848282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34560a6-0d97-4080-bd6e-d73626a56c99 admin admin] PATH_INFO: `/v3/users/041ac6a41f814755af6874f12bfdd960` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:55.851065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34560a6-0d97-4080-bd6e-d73626a56c99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=041ac6a41f814755af6874f12bfdd960)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:55.855746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34560a6-0d97-4080-bd6e-d73626a56c99 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:56.025187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f34560a6-0d97-4080-bd6e-d73626a56c99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '041ac6a41f814755af6874f12bfdd960'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:56.052390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f34560a6-0d97-4080-bd6e-d73626a56c99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '041ac6a41f814755af6874f12bfdd960'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:56.055010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f34560a6-0d97-4080-bd6e-d73626a56c99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '041ac6a41f814755af6874f12bfdd960'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:56.077308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f34560a6-0d97-4080-bd6e-d73626a56c99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '041ac6a41f814755af6874f12bfdd960'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:56.113228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f34560a6-0d97-4080-bd6e-d73626a56c99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '041ac6a41f814755af6874f12bfdd960'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:56.156673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 932/1865] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:55 2022] DELETE /identity/v3/users/041ac6a41f814755af6874f12bfdd960 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:56.253352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:56.255689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:56.294635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343da88-a972-495c-9bdc-7b2bfe069927 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:56.433028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:56.433973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:56.434451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:56.435038 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a admin admin] PATH_INFO: `/v3/projects/8a89eb0302d84788baa565fe287e4a6d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:56.437831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a89eb0302d84788baa565fe287e4a6d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:56.442385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:56.567965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a89eb0302d84788baa565fe287e4a6d', 'request_id': 'req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:56.587560 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a89eb0302d84788baa565fe287e4a6d', 'request_id': 'req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:56.598145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a89eb0302d84788baa565fe287e4a6d', 'request_id': 'req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:56.633310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a admin admin] The token cache is being invalidate because project 8a89eb0302d84788baa565fe287e4a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:56.634854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:56.636355 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ca6aef69-9ef6-4ab0-baf6-ee4c8682f32a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:56.646962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 934/1866] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:56 2022] DELETE /identity/v3/projects/8a89eb0302d84788baa565fe287e4a6d => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:23:56.648951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dea2c0-779d-4d14-832b-c736cb30b437 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:56.649339 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dea2c0-779d-4d14-832b-c736cb30b437 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:56.649636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dea2c0-779d-4d14-832b-c736cb30b437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:56.687720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a2dea2c0-779d-4d14-832b-c736cb30b437 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:56.688654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a2dea2c0-779d-4d14-832b-c736cb30b437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:56.729850 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:56.820365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 933/1867] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:23:56.826109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac35fe-86f1-4951-861f-a1ce636a75ca None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:56.826302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac35fe-86f1-4951-861f-a1ce636a75ca None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:56.826637 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaac35fe-86f1-4951-861f-a1ce636a75ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:56.896068 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-aaac35fe-86f1-4951-861f-a1ce636a75ca None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:56.898434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-aaac35fe-86f1-4951-861f-a1ce636a75ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:56.918365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 935/1868] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:56.920664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dea2c0-779d-4d14-832b-c736cb30b437 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:56.942112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dea2c0-779d-4d14-832b-c736cb30b437 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:56.958395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f24434-4ce5-4077-8f07-220282c34f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:56.958395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f24434-4ce5-4077-8f07-220282c34f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:56.958395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f24434-4ce5-4077-8f07-220282c34f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:56.958395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f24434-4ce5-4077-8f07-220282c34f93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:56.958716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f24434-4ce5-4077-8f07-220282c34f93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:23:56.958716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f24434-4ce5-4077-8f07-220282c34f93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:56.959697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f24434-4ce5-4077-8f07-220282c34f93 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:56.975028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac35fe-86f1-4951-861f-a1ce636a75ca None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:56.977391 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-aaac35fe-86f1-4951-861f-a1ce636a75ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:56.990845 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 934/1869] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:23:57.008548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f24434-4ce5-4077-8f07-220282c34f93 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:57.029567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f24434-4ce5-4077-8f07-220282c34f93 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:57.035724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc9e68-d2a9-4b51-b565-d1e116fbbced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:57.037116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc9e68-d2a9-4b51-b565-d1e116fbbced admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:57.037496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc9e68-d2a9-4b51-b565-d1e116fbbced admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:57.037844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc9e68-d2a9-4b51-b565-d1e116fbbced admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:57.038603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc9e68-d2a9-4b51-b565-d1e116fbbced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:57.042032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc9e68-d2a9-4b51-b565-d1e116fbbced admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:57.047837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaac35fe-86f1-4951-861f-a1ce636a75ca None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:57.056749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:57.058790 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:57.059085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:57.059364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] PATH_INFO: `/v3/projects/c8ee5ae41ae748fe8a2604c01415ecec` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:57.062176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ee5ae41ae748fe8a2604c01415ecec)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:57.065003 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:57.106124 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 935/1870] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:23:56 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:23:57.109482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc9e68-d2a9-4b51-b565-d1e116fbbced admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:57.177570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ee5ae41ae748fe8a2604c01415ecec', 'request_id': 'req-45fb29b9-c32f-4a52-b80e-a1e46dddc419'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:57.191725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ee5ae41ae748fe8a2604c01415ecec', 'request_id': 'req-45fb29b9-c32f-4a52-b80e-a1e46dddc419'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:57.208338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:57.213685 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ee5ae41ae748fe8a2604c01415ecec', 'request_id': 'req-45fb29b9-c32f-4a52-b80e-a1e46dddc419'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:57.248232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc9e68-d2a9-4b51-b565-d1e116fbbced admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:57.262387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] The token cache is being invalidate because project c8ee5ae41ae748fe8a2604c01415ecec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:23:57.262989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:57.267682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:57.272071 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 936/1871] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:56 2022] DELETE /identity/v3/projects/c8ee5ae41ae748fe8a2604c01415ecec => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:57.276800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:57.280841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:57.405579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16be10f-c4e9-4032-b686-c04db567a960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:57.406309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16be10f-c4e9-4032-b686-c04db567a960 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:57.406762 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16be10f-c4e9-4032-b686-c04db567a960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:57.407158 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16be10f-c4e9-4032-b686-c04db567a960 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:57.407878 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16be10f-c4e9-4032-b686-c04db567a960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:57.411519 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16be10f-c4e9-4032-b686-c04db567a960 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:57.435139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb29b9-c32f-4a52-b80e-a1e46dddc419 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:57.442115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 936/1872] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:23:57 2022] POST /identity/v3/users => generated 286 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:23:57.444839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51a3f95-3fb9-4681-bba0-3b6d11d293fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:57.445438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16be10f-c4e9-4032-b686-c04db567a960 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:57.445944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a3f95-3fb9-4681-bba0-3b6d11d293fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:57.447041 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a3f95-3fb9-4681-bba0-3b6d11d293fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:57.447592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51a3f95-3fb9-4681-bba0-3b6d11d293fb admin admin] PATH_INFO: `/v3/users/a9d26d63903c4b25835a580ca295a78a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:57.448500 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b16be10f-c4e9-4032-b686-c04db567a960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:57.449493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a3f95-3fb9-4681-bba0-3b6d11d293fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9d26d63903c4b25835a580ca295a78a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:57.453164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51a3f95-3fb9-4681-bba0-3b6d11d293fb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:57.472704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16be10f-c4e9-4032-b686-c04db567a960 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:57.479314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a801b32-ce9d-4ee0-b429-748fdbcb1eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:57.480063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a801b32-ce9d-4ee0-b429-748fdbcb1eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:57.489458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a801b32-ce9d-4ee0-b429-748fdbcb1eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:57.489578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a801b32-ce9d-4ee0-b429-748fdbcb1eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:57.491506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a801b32-ce9d-4ee0-b429-748fdbcb1eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:57.492210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a801b32-ce9d-4ee0-b429-748fdbcb1eb2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:57.519285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 937/1873] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:57.524141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a801b32-ce9d-4ee0-b429-748fdbcb1eb2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:57.537922 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a801b32-ce9d-4ee0-b429-748fdbcb1eb2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:57.545824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e5ff3-09c2-4e58-8b0f-82908b387a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:57.546636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e5ff3-09c2-4e58-8b0f-82908b387a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:57.546968 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e5ff3-09c2-4e58-8b0f-82908b387a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:57.547369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e5ff3-09c2-4e58-8b0f-82908b387a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:57.547974 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e5ff3-09c2-4e58-8b0f-82908b387a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:57.550474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889e5ff3-09c2-4e58-8b0f-82908b387a55 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:57.566235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f51a3f95-3fb9-4681-bba0-3b6d11d293fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9d26d63903c4b25835a580ca295a78a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:57.568508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 938/1874] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:23:57.573832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e5ff3-09c2-4e58-8b0f-82908b387a55 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:57.586510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:57.586510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e5ff3-09c2-4e58-8b0f-82908b387a55 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:57.589900 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f51a3f95-3fb9-4681-bba0-3b6d11d293fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9d26d63903c4b25835a580ca295a78a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:57.613612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06cceb-33b0-435f-9e60-df0f0babc8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:57.617926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06cceb-33b0-435f-9e60-df0f0babc8f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:57.619093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06cceb-33b0-435f-9e60-df0f0babc8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:57.619587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06cceb-33b0-435f-9e60-df0f0babc8f3 admin admin] PATH_INFO: `/v3/projects/34e4ec58f4ec4fcca2469da8b7137bf0/users/5119320f7b134c428df9e77bacae7b46/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:57.662177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f51a3f95-3fb9-4681-bba0-3b6d11d293fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9d26d63903c4b25835a580ca295a78a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:57.666284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f51a3f95-3fb9-4681-bba0-3b6d11d293fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9d26d63903c4b25835a580ca295a78a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:57.681310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f51a3f95-3fb9-4681-bba0-3b6d11d293fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9d26d63903c4b25835a580ca295a78a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:57.683027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06cceb-33b0-435f-9e60-df0f0babc8f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34e4ec58f4ec4fcca2469da8b7137bf0, user_id=5119320f7b134c428df9e77bacae7b46, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:57.695726 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc06cceb-33b0-435f-9e60-df0f0babc8f3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:57.709490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 937/1875] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:57 2022] DELETE /identity/v3/users/a9d26d63903c4b25835a580ca295a78a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:57.764021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 939/1876] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:23:57 2022] PUT /identity/v3/projects/34e4ec58f4ec4fcca2469da8b7137bf0/users/5119320f7b134c428df9e77bacae7b46/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:57.778515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d20a13-aa32-4567-8863-a6e39a1b82fd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:57.778900 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d20a13-aa32-4567-8863-a6e39a1b82fd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:57.779183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d20a13-aa32-4567-8863-a6e39a1b82fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:57.854985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d3d20a13-aa32-4567-8863-a6e39a1b82fd None None] MFA Rules not processed for user `5119320f7b134c428df9e77bacae7b46`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:57.856638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d3d20a13-aa32-4567-8863-a6e39a1b82fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:58.022604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:23:58.039914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 938/1877] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:58.120692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06cceb-33b0-435f-9e60-df0f0babc8f3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:58.157141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-dc06cceb-33b0-435f-9e60-df0f0babc8f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:58.349297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06cceb-33b0-435f-9e60-df0f0babc8f3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:58.359631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf34109-2882-48cf-ae14-a5120070eb7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:58.360921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf34109-2882-48cf-ae14-a5120070eb7a service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:58.361550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf34109-2882-48cf-ae14-a5120070eb7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:58.361987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf34109-2882-48cf-ae14-a5120070eb7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:58.382742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf34109-2882-48cf-ae14-a5120070eb7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:58.386815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf34109-2882-48cf-ae14-a5120070eb7a service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:58.404605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 940/1878] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:58.431030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d20a13-aa32-4567-8863-a6e39a1b82fd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:58.432533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d3d20a13-aa32-4567-8863-a6e39a1b82fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:58.543168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d20a13-aa32-4567-8863-a6e39a1b82fd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:58.554862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:58.556169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:58.556687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:58.557169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] PATH_INFO: `/v3/users/5119320f7b134c428df9e77bacae7b46` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:58.559623 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5119320f7b134c428df9e77bacae7b46)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:58.563460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:58.657933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5119320f7b134c428df9e77bacae7b46'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:58.672276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5119320f7b134c428df9e77bacae7b46'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:58.694390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5119320f7b134c428df9e77bacae7b46'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:58.695525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5119320f7b134c428df9e77bacae7b46'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:58.706482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5119320f7b134c428df9e77bacae7b46'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:23:58.715284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 939/1879] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:23:58 2022] DELETE /identity/v3/users/5119320f7b134c428df9e77bacae7b46 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:23:58.721257 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074ce14-5bb6-4b2d-a3b9-9535d0ed24b9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:58.721775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074ce14-5bb6-4b2d-a3b9-9535d0ed24b9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:58.722179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074ce14-5bb6-4b2d-a3b9-9535d0ed24b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:58.749789 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3074ce14-5bb6-4b2d-a3b9-9535d0ed24b9 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:23:58.751135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3074ce14-5bb6-4b2d-a3b9-9535d0ed24b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:58.862228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 941/1880] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:23:58.875738 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:58.877679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:58.975114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e888ac-de6c-48b9-b66f-6f8b49a059c8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:58.980581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30574f-2594-4c63-834c-077b6fe4b1fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:58.981823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30574f-2594-4c63-834c-077b6fe4b1fe service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:58.982410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30574f-2594-4c63-834c-077b6fe4b1fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:58.982888 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30574f-2594-4c63-834c-077b6fe4b1fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:58.996574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30574f-2594-4c63-834c-077b6fe4b1fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:58.999188 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30574f-2594-4c63-834c-077b6fe4b1fe service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:23:59.015111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 940/1881] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:23:59.792413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074ce14-5bb6-4b2d-a3b9-9535d0ed24b9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:23:59.795707 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3074ce14-5bb6-4b2d-a3b9-9535d0ed24b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:23:59.855131 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074ce14-5bb6-4b2d-a3b9-9535d0ed24b9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:23:59.864427 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:23:59.865972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:23:59.866768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:23:59.867246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] PATH_INFO: `/v3/projects/cfae3a96d3104d06b71c3d7e62d38bad` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:23:59.869997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfae3a96d3104d06b71c3d7e62d38bad)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:23:59.873853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:00.017420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfae3a96d3104d06b71c3d7e62d38bad', 'request_id': 'req-997cf27b-23f2-43a9-93a7-246298c21fe4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:00.040913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfae3a96d3104d06b71c3d7e62d38bad', 'request_id': 'req-997cf27b-23f2-43a9-93a7-246298c21fe4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:00.067798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfae3a96d3104d06b71c3d7e62d38bad', 'request_id': 'req-997cf27b-23f2-43a9-93a7-246298c21fe4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:00.094565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] The token cache is being invalidate because project cfae3a96d3104d06b71c3d7e62d38bad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:00.095085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:00.096369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:00.109676 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 942/1882] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:23:59 2022] DELETE /identity/v3/projects/cfae3a96d3104d06b71c3d7e62d38bad => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:00.111594 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cf27c-e495-4ede-b99a-d19b9d8cd9fc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:00.113115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cf27c-e495-4ede-b99a-d19b9d8cd9fc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:00.113840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cf27c-e495-4ede-b99a-d19b9d8cd9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:00.160751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7b9cf27c-e495-4ede-b99a-d19b9d8cd9fc None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:00.164029 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7b9cf27c-e495-4ede-b99a-d19b9d8cd9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:00.214448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:00.256069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:00.266743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:00.287390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:00.320657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 941/1883] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:00.332482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756dfbe3-06a5-41a1-9131-9f221d8c928f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:00.333240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756dfbe3-06a5-41a1-9131-9f221d8c928f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:00.333857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756dfbe3-06a5-41a1-9131-9f221d8c928f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:00.370457 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-756dfbe3-06a5-41a1-9131-9f221d8c928f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:00.374373 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-756dfbe3-06a5-41a1-9131-9f221d8c928f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:00.393692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 942/1884] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:00.395843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756dfbe3-06a5-41a1-9131-9f221d8c928f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:00.417076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756dfbe3-06a5-41a1-9131-9f221d8c928f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:00.438086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01ef1c-e19e-4ea1-8dd3-3d7032ca25b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:00.438086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01ef1c-e19e-4ea1-8dd3-3d7032ca25b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:00.438751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cf27b-23f2-43a9-93a7-246298c21fe4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:00.438932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01ef1c-e19e-4ea1-8dd3-3d7032ca25b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:00.438932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01ef1c-e19e-4ea1-8dd3-3d7032ca25b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:00.438932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01ef1c-e19e-4ea1-8dd3-3d7032ca25b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:24:00.438932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01ef1c-e19e-4ea1-8dd3-3d7032ca25b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:00.438932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01ef1c-e19e-4ea1-8dd3-3d7032ca25b7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:00.455223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96642e9-4638-42f2-bcb0-70a8555fe197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:00.457391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96642e9-4638-42f2-bcb0-70a8555fe197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:00.458162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96642e9-4638-42f2-bcb0-70a8555fe197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:00.459185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 943/1885] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:24:00.464320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96642e9-4638-42f2-bcb0-70a8555fe197 admin admin] PATH_INFO: `/v3/projects/34e4ec58f4ec4fcca2469da8b7137bf0` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:00.467158 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01ef1c-e19e-4ea1-8dd3-3d7032ca25b7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:00.469246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96642e9-4638-42f2-bcb0-70a8555fe197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34e4ec58f4ec4fcca2469da8b7137bf0)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:00.477399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96642e9-4638-42f2-bcb0-70a8555fe197 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:00.490581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01ef1c-e19e-4ea1-8dd3-3d7032ca25b7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:00.498634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f24e1-7d48-4318-a617-4ad06338f631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:00.499888 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f24e1-7d48-4318-a617-4ad06338f631 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:00.500508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f24e1-7d48-4318-a617-4ad06338f631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:00.501211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7f24e1-7d48-4318-a617-4ad06338f631 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:00.502592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f24e1-7d48-4318-a617-4ad06338f631 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:00.506892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7f24e1-7d48-4318-a617-4ad06338f631 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:00.565337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 944/1886] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:00 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:00.570999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f24e1-7d48-4318-a617-4ad06338f631 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:00.632884 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e96642e9-4638-42f2-bcb0-70a8555fe197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34e4ec58f4ec4fcca2469da8b7137bf0', 'request_id': 'req-e96642e9-4638-42f2-bcb0-70a8555fe197'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:00.655012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e96642e9-4638-42f2-bcb0-70a8555fe197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34e4ec58f4ec4fcca2469da8b7137bf0', 'request_id': 'req-e96642e9-4638-42f2-bcb0-70a8555fe197'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:00.674286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7f24e1-7d48-4318-a617-4ad06338f631 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:00.680509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e96642e9-4638-42f2-bcb0-70a8555fe197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34e4ec58f4ec4fcca2469da8b7137bf0', 'request_id': 'req-e96642e9-4638-42f2-bcb0-70a8555fe197'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:00.717290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e96642e9-4638-42f2-bcb0-70a8555fe197 admin admin] The token cache is being invalidate because project 34e4ec58f4ec4fcca2469da8b7137bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:00.718195 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e96642e9-4638-42f2-bcb0-70a8555fe197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:00.721016 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e96642e9-4638-42f2-bcb0-70a8555fe197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:00.725003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 943/1887] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:00 2022] DELETE /identity/v3/projects/34e4ec58f4ec4fcca2469da8b7137bf0 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:00.733988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f43c67-f1ec-4919-8670-09ae0a662df1 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:00.733988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f43c67-f1ec-4919-8670-09ae0a662df1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:00.739426 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f43c67-f1ec-4919-8670-09ae0a662df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:00.803838 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491ecd8-7d66-4100-be53-880ac200e4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:00.805032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491ecd8-7d66-4100-be53-880ac200e4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:00.805572 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491ecd8-7d66-4100-be53-880ac200e4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:00.806084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491ecd8-7d66-4100-be53-880ac200e4f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:00.806421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-31f43c67-f1ec-4919-8670-09ae0a662df1 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:00.807284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491ecd8-7d66-4100-be53-880ac200e4f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:00.809517 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-31f43c67-f1ec-4919-8670-09ae0a662df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:00.811829 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491ecd8-7d66-4100-be53-880ac200e4f1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:00.819353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 944/1888] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:00.829168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fd297-ba13-48c6-9fea-dc8c633ae700 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:00.830001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fd297-ba13-48c6-9fea-dc8c633ae700 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:00.830745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fd297-ba13-48c6-9fea-dc8c633ae700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:00.845748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 945/1889] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:00 2022] POST /identity/v3/users => generated 282 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:00.850122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491ecd8-7d66-4100-be53-880ac200e4f1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:00.851937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f491ecd8-7d66-4100-be53-880ac200e4f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:00.867597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491ecd8-7d66-4100-be53-880ac200e4f1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:00.873966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090691e-867a-4077-bfc1-0dbdae3b32d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:00.875043 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090691e-867a-4077-bfc1-0dbdae3b32d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:00.875043 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090691e-867a-4077-bfc1-0dbdae3b32d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:00.875539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090691e-867a-4077-bfc1-0dbdae3b32d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:00.876427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090691e-867a-4077-bfc1-0dbdae3b32d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:00.878346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090691e-867a-4077-bfc1-0dbdae3b32d1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:00.888281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1e9fd297-ba13-48c6-9fea-dc8c633ae700 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:00.890685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1e9fd297-ba13-48c6-9fea-dc8c633ae700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:00.896621 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 946/1890] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:00.897860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090691e-867a-4077-bfc1-0dbdae3b32d1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:00.916978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090691e-867a-4077-bfc1-0dbdae3b32d1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:00.924968 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 945/1891] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:00.927810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fd297-ba13-48c6-9fea-dc8c633ae700 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:00.934169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fa71e-e70a-422c-8c7d-6a93ded0d2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:00.935369 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fa71e-e70a-422c-8c7d-6a93ded0d2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:00.935880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fa71e-e70a-422c-8c7d-6a93ded0d2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:00.936477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6fa71e-e70a-422c-8c7d-6a93ded0d2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:00.937660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fa71e-e70a-422c-8c7d-6a93ded0d2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:00.939775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6fa71e-e70a-422c-8c7d-6a93ded0d2a0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:00.942229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:00.954395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fd297-ba13-48c6-9fea-dc8c633ae700 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:00.961289 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 947/1892] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:00.962361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fa71e-e70a-422c-8c7d-6a93ded0d2a0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:00.970426 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29f46a-b93b-4175-a076-1628b3acde37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:00.973086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29f46a-b93b-4175-a076-1628b3acde37 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:00.973941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29f46a-b93b-4175-a076-1628b3acde37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:00.975669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29f46a-b93b-4175-a076-1628b3acde37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:00.977263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29f46a-b93b-4175-a076-1628b3acde37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:24:00.978350 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29f46a-b93b-4175-a076-1628b3acde37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:00.979122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6fa71e-e70a-422c-8c7d-6a93ded0d2a0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:00.985795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43aaf3f-3c06-40f2-b16a-a439b4019484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:00.987633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43aaf3f-3c06-40f2-b16a-a439b4019484 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:00.988044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43aaf3f-3c06-40f2-b16a-a439b4019484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:00.988424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43aaf3f-3c06-40f2-b16a-a439b4019484 admin admin] PATH_INFO: `/v3/projects/cb83e7659d7e434fb1b4fabc703d0506/users/7582a7c5588b42acbdf34b63df997bf8/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:00.993797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29f46a-b93b-4175-a076-1628b3acde37 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:01.023530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43aaf3f-3c06-40f2-b16a-a439b4019484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb83e7659d7e434fb1b4fabc703d0506, user_id=7582a7c5588b42acbdf34b63df997bf8, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:01.030826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 946/1893] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:24:01.031052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43aaf3f-3c06-40f2-b16a-a439b4019484 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:01.032737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29f46a-b93b-4175-a076-1628b3acde37 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:01.074376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29f46a-b93b-4175-a076-1628b3acde37 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:01.078192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 948/1894] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:00 2022] PUT /identity/v3/projects/cb83e7659d7e434fb1b4fabc703d0506/users/7582a7c5588b42acbdf34b63df997bf8/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:01.087196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b737515-984a-4c30-9bcd-a4c7e3f99241 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:01.087493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b737515-984a-4c30-9bcd-a4c7e3f99241 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:01.087759 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b737515-984a-4c30-9bcd-a4c7e3f99241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:01.160235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4b737515-984a-4c30-9bcd-a4c7e3f99241 None None] MFA Rules not processed for user `7582a7c5588b42acbdf34b63df997bf8`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:01.161536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4b737515-984a-4c30-9bcd-a4c7e3f99241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:01.282210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4474cc-75c3-4c94-ab8b-f7e94d67f043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:01.283174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4474cc-75c3-4c94-ab8b-f7e94d67f043 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:01.283934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4474cc-75c3-4c94-ab8b-f7e94d67f043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:01.284403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4474cc-75c3-4c94-ab8b-f7e94d67f043 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:01.285784 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4474cc-75c3-4c94-ab8b-f7e94d67f043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:01.289584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4474cc-75c3-4c94-ab8b-f7e94d67f043 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:01.344015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:01.363318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 947/1895] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:01 2022] POST /identity/v3/projects => generated 375 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:01.368917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4474cc-75c3-4c94-ab8b-f7e94d67f043 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:01.502702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 949/1896] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:01.534589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4474cc-75c3-4c94-ab8b-f7e94d67f043 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:01.540049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3e348-dc17-446c-b7bc-bd6bb754e3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:01.541348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3e348-dc17-446c-b7bc-bd6bb754e3f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:01.541754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3e348-dc17-446c-b7bc-bd6bb754e3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:01.542249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e3e348-dc17-446c-b7bc-bd6bb754e3f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:01.543103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3e348-dc17-446c-b7bc-bd6bb754e3f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:01.545287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e3e348-dc17-446c-b7bc-bd6bb754e3f0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:01.601132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 948/1897] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:01 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:01.604432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b737515-984a-4c30-9bcd-a4c7e3f99241 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:01.643943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b737515-984a-4c30-9bcd-a4c7e3f99241 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:01.677470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3e348-dc17-446c-b7bc-bd6bb754e3f0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:01.686221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-74e3e348-dc17-446c-b7bc-bd6bb754e3f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:01.688148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b94aa-2cbd-4cc3-ad4b-fdf39d885fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:01.691178 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b94aa-2cbd-4cc3-ad4b-fdf39d885fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:01.691409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b94aa-2cbd-4cc3-ad4b-fdf39d885fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:01.691695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b94aa-2cbd-4cc3-ad4b-fdf39d885fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:01.692443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b94aa-2cbd-4cc3-ad4b-fdf39d885fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:01.695224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b94aa-2cbd-4cc3-ad4b-fdf39d885fb9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:01.757627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 950/1898] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:01.772324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b94aa-2cbd-4cc3-ad4b-fdf39d885fb9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:01.814266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b94aa-2cbd-4cc3-ad4b-fdf39d885fb9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:01.822222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767eb8a-c958-425f-83e2-649cd6684ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:01.822893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767eb8a-c958-425f-83e2-649cd6684ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:01.823176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767eb8a-c958-425f-83e2-649cd6684ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:01.823485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767eb8a-c958-425f-83e2-649cd6684ee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:01.824210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767eb8a-c958-425f-83e2-649cd6684ee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:01.826005 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767eb8a-c958-425f-83e2-649cd6684ee0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:01.842723 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 951/1899] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:01.850439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767eb8a-c958-425f-83e2-649cd6684ee0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:01.863569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767eb8a-c958-425f-83e2-649cd6684ee0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:01.872327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637a256-d479-40b9-89c5-33fe07475f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:01.873282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637a256-d479-40b9-89c5-33fe07475f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:01.873804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637a256-d479-40b9-89c5-33fe07475f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:01.874441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637a256-d479-40b9-89c5-33fe07475f79 admin admin] PATH_INFO: `/v3/projects/faf59212b8e34290a86148da9cf45de2/users/2cc4605484de43aa8c3b8b7383d20e32/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:01.908100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637a256-d479-40b9-89c5-33fe07475f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf59212b8e34290a86148da9cf45de2, user_id=2cc4605484de43aa8c3b8b7383d20e32, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:01.918208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637a256-d479-40b9-89c5-33fe07475f79 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:01.933418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e3e348-dc17-446c-b7bc-bd6bb754e3f0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:01.949365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656b745-0d12-40cf-bccd-cdecc20f801f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:01.950347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656b745-0d12-40cf-bccd-cdecc20f801f service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:01.950974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 952/1900] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:01 2022] PUT /identity/v3/projects/faf59212b8e34290a86148da9cf45de2/users/2cc4605484de43aa8c3b8b7383d20e32/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:01.951211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656b745-0d12-40cf-bccd-cdecc20f801f service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:01.951609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656b745-0d12-40cf-bccd-cdecc20f801f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:01.965168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bacc6d-67a6-4bea-b38d-270bb43ad1e7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:01.965665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bacc6d-67a6-4bea-b38d-270bb43ad1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:01.968936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bacc6d-67a6-4bea-b38d-270bb43ad1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:02.095919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e7bacc6d-67a6-4bea-b38d-270bb43ad1e7 None None] MFA Rules not processed for user `2cc4605484de43aa8c3b8b7383d20e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:02.098557 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e7bacc6d-67a6-4bea-b38d-270bb43ad1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:02.148995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656b745-0d12-40cf-bccd-cdecc20f801f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:02.151482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656b745-0d12-40cf-bccd-cdecc20f801f service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:02.165902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 949/1901] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:02.241976 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 953/1902] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:02.382775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656b745-0d12-40cf-bccd-cdecc20f801f service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:02.385688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f656b745-0d12-40cf-bccd-cdecc20f801f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:02.406024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656b745-0d12-40cf-bccd-cdecc20f801f service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:02.529358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:02.551865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b18885-1b0d-49fb-ad24-73da9fdcfa82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:02.552525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b18885-1b0d-49fb-ad24-73da9fdcfa82 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:02.553025 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b18885-1b0d-49fb-ad24-73da9fdcfa82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:02.553309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b18885-1b0d-49fb-ad24-73da9fdcfa82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:02.568886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b18885-1b0d-49fb-ad24-73da9fdcfa82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:02.570516 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b18885-1b0d-49fb-ad24-73da9fdcfa82 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:02.605364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 950/1903] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:06.949447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bacc6d-67a6-4bea-b38d-270bb43ad1e7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:06.958787 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:07.084051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bacc6d-67a6-4bea-b38d-270bb43ad1e7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:07.115694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:07.116489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:07.117117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:07.122114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] PATH_INFO: `/v3/users/2cc4605484de43aa8c3b8b7383d20e32` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:07.125010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cc4605484de43aa8c3b8b7383d20e32)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:07.127881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:07.215980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cc4605484de43aa8c3b8b7383d20e32'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:07.232954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cc4605484de43aa8c3b8b7383d20e32'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:07.251825 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cc4605484de43aa8c3b8b7383d20e32'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:07.252659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cc4605484de43aa8c3b8b7383d20e32'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:07.263115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cc4605484de43aa8c3b8b7383d20e32'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:07.275413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 954/1904] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:06 2022] DELETE /identity/v3/users/2cc4605484de43aa8c3b8b7383d20e32 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:07.280681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90a724-efd2-4648-a26b-b4bf14efc541 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:07.281769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90a724-efd2-4648-a26b-b4bf14efc541 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:07.282188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90a724-efd2-4648-a26b-b4bf14efc541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:07.331625 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4e90a724-efd2-4648-a26b-b4bf14efc541 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:07.333892 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4e90a724-efd2-4648-a26b-b4bf14efc541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:07.443857 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 951/1905] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:07.461428 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:07.463091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:07.608752 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b84db-2916-4745-a74c-9b09459ccc57 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:07.616534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29463c5-2a3f-4c11-af9b-ce4352a957d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:07.616844 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29463c5-2a3f-4c11-af9b-ce4352a957d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:07.617245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29463c5-2a3f-4c11-af9b-ce4352a957d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:07.618619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29463c5-2a3f-4c11-af9b-ce4352a957d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:07.635272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29463c5-2a3f-4c11-af9b-ce4352a957d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:07.638205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29463c5-2a3f-4c11-af9b-ce4352a957d9 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:07.654784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 955/1906] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:24:08.293492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90a724-efd2-4648-a26b-b4bf14efc541 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:08.295956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4e90a724-efd2-4648-a26b-b4bf14efc541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:08.318542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90a724-efd2-4648-a26b-b4bf14efc541 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:08.328407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f002097-0ccb-464b-a855-d9241c18978a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:08.329581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f002097-0ccb-464b-a855-d9241c18978a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:08.330320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f002097-0ccb-464b-a855-d9241c18978a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:08.330912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f002097-0ccb-464b-a855-d9241c18978a admin admin] PATH_INFO: `/v3/projects/faf59212b8e34290a86148da9cf45de2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:08.333428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f002097-0ccb-464b-a855-d9241c18978a admin admin] RBAC: Authorizing `identity:delete_project(project_id=faf59212b8e34290a86148da9cf45de2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:08.338490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f002097-0ccb-464b-a855-d9241c18978a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:08.438319 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5f002097-0ccb-464b-a855-d9241c18978a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faf59212b8e34290a86148da9cf45de2', 'request_id': 'req-5f002097-0ccb-464b-a855-d9241c18978a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:08.457579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5f002097-0ccb-464b-a855-d9241c18978a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faf59212b8e34290a86148da9cf45de2', 'request_id': 'req-5f002097-0ccb-464b-a855-d9241c18978a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:08.478576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5f002097-0ccb-464b-a855-d9241c18978a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faf59212b8e34290a86148da9cf45de2', 'request_id': 'req-5f002097-0ccb-464b-a855-d9241c18978a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:08.505103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5f002097-0ccb-464b-a855-d9241c18978a admin admin] The token cache is being invalidate because project faf59212b8e34290a86148da9cf45de2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:08.505723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5f002097-0ccb-464b-a855-d9241c18978a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:08.507685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5f002097-0ccb-464b-a855-d9241c18978a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:08.515202 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 952/1907] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:08 2022] DELETE /identity/v3/projects/faf59212b8e34290a86148da9cf45de2 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:08.523676 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8823-f56f-4f87-bd54-df3f600ad8ee None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:08.524452 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8823-f56f-4f87-bd54-df3f600ad8ee None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:08.524666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8823-f56f-4f87-bd54-df3f600ad8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:08.566698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-203e8823-f56f-4f87-bd54-df3f600ad8ee None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:08.569498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-203e8823-f56f-4f87-bd54-df3f600ad8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:08.619978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:08.733865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 956/1908] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:08.742472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4833919-c120-4bb5-b30d-46dc269ade38 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:08.742897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4833919-c120-4bb5-b30d-46dc269ade38 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:08.743165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4833919-c120-4bb5-b30d-46dc269ade38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:08.785023 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a4833919-c120-4bb5-b30d-46dc269ade38 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:08.786666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a4833919-c120-4bb5-b30d-46dc269ade38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:08.801810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8823-f56f-4f87-bd54-df3f600ad8ee None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:08.807584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 953/1909] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:08.829645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8823-f56f-4f87-bd54-df3f600ad8ee None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:08.858759 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6532b-6320-4f4d-9a29-3975979928db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:08.860306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6532b-6320-4f4d-9a29-3975979928db admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:08.861772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6532b-6320-4f4d-9a29-3975979928db admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:08.864523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6532b-6320-4f4d-9a29-3975979928db admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:08.870532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6532b-6320-4f4d-9a29-3975979928db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:24:08.871416 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6532b-6320-4f4d-9a29-3975979928db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:08.875240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6532b-6320-4f4d-9a29-3975979928db admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:08.925660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 957/1910] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:24:08.928232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4833919-c120-4bb5-b30d-46dc269ade38 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:08.944787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4833919-c120-4bb5-b30d-46dc269ade38 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:08.966702 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25742a4f-e074-48f1-afb3-7b5177732c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:08.967817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25742a4f-e074-48f1-afb3-7b5177732c39 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:08.968307 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25742a4f-e074-48f1-afb3-7b5177732c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:08.968789 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25742a4f-e074-48f1-afb3-7b5177732c39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:08.970249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25742a4f-e074-48f1-afb3-7b5177732c39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:08.973993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25742a4f-e074-48f1-afb3-7b5177732c39 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:09.067144 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 954/1911] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:08 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:09.072345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6532b-6320-4f4d-9a29-3975979928db admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:09.095256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6532b-6320-4f4d-9a29-3975979928db admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:09.241004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb3059-53ba-4dd8-b6be-17b561d37fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:09.241887 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb3059-53ba-4dd8-b6be-17b561d37fdf admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:09.243067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb3059-53ba-4dd8-b6be-17b561d37fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:09.243500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb3059-53ba-4dd8-b6be-17b561d37fdf admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:09.244501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb3059-53ba-4dd8-b6be-17b561d37fdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:09.246773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb3059-53ba-4dd8-b6be-17b561d37fdf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:09.293442 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 958/1912] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:09 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:09.299434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25742a4f-e074-48f1-afb3-7b5177732c39 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:09.318066 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25742a4f-e074-48f1-afb3-7b5177732c39 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:09.327704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d9429-70f5-4959-9c70-7d9c5e4a61a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:09.328349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d9429-70f5-4959-9c70-7d9c5e4a61a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:09.329004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d9429-70f5-4959-9c70-7d9c5e4a61a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:09.329589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9d9429-70f5-4959-9c70-7d9c5e4a61a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:09.334717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d9429-70f5-4959-9c70-7d9c5e4a61a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:09.343672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9d9429-70f5-4959-9c70-7d9c5e4a61a0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:09.379630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb3059-53ba-4dd8-b6be-17b561d37fdf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:09.382079 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 955/1913] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:09.402458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb3059-53ba-4dd8-b6be-17b561d37fdf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:09.415641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f0ef2-161f-4fe3-b859-c6ac9f729145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:09.416705 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f0ef2-161f-4fe3-b859-c6ac9f729145 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:09.417252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f0ef2-161f-4fe3-b859-c6ac9f729145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:09.419777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f0ef2-161f-4fe3-b859-c6ac9f729145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:09.421135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f0ef2-161f-4fe3-b859-c6ac9f729145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:09.425387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f0ef2-161f-4fe3-b859-c6ac9f729145 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:09.470909 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 959/1914] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:09.487554 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d9429-70f5-4959-9c70-7d9c5e4a61a0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:09.543277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9d9429-70f5-4959-9c70-7d9c5e4a61a0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:09.570137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f42ce2-20f5-4e33-9eaa-f1456ba9bf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:09.570880 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f42ce2-20f5-4e33-9eaa-f1456ba9bf5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:09.571168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f42ce2-20f5-4e33-9eaa-f1456ba9bf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:09.571448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f42ce2-20f5-4e33-9eaa-f1456ba9bf5f admin admin] PATH_INFO: `/v3/projects/20c1349283a747ac8582022a84887e04/users/24e4e8f827e34b44a457a5b4218b22f1/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:09.615382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f42ce2-20f5-4e33-9eaa-f1456ba9bf5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=20c1349283a747ac8582022a84887e04, user_id=24e4e8f827e34b44a457a5b4218b22f1, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:09.625972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f42ce2-20f5-4e33-9eaa-f1456ba9bf5f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:09.679559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 956/1915] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:09 2022] PUT /identity/v3/projects/20c1349283a747ac8582022a84887e04/users/24e4e8f827e34b44a457a5b4218b22f1/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:09.709694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bfc97-6e54-458b-9f99-bf25b8ca6d54 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:09.710422 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bfc97-6e54-458b-9f99-bf25b8ca6d54 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:09.710717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bfc97-6e54-458b-9f99-bf25b8ca6d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:09.791210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9c9bfc97-6e54-458b-9f99-bf25b8ca6d54 None None] MFA Rules not processed for user `24e4e8f827e34b44a457a5b4218b22f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:09.792928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9c9bfc97-6e54-458b-9f99-bf25b8ca6d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:09.933104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 960/1916] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:09.939029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f42ce2-20f5-4e33-9eaa-f1456ba9bf5f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:09.949053 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:10.113885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f42ce2-20f5-4e33-9eaa-f1456ba9bf5f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:10.121216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731af04e-0591-4fb7-8b38-640a2776a456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:10.121770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731af04e-0591-4fb7-8b38-640a2776a456 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:10.122157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731af04e-0591-4fb7-8b38-640a2776a456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:10.122509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731af04e-0591-4fb7-8b38-640a2776a456 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:10.123245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731af04e-0591-4fb7-8b38-640a2776a456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:10.125856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731af04e-0591-4fb7-8b38-640a2776a456 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:10.178309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 957/1917] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:09 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:10.183352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bfc97-6e54-458b-9f99-bf25b8ca6d54 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:10.201529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bfc97-6e54-458b-9f99-bf25b8ca6d54 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:10.283649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:10.489754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb1b51-e322-4769-9404-8625fc268f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:10.489901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb1b51-e322-4769-9404-8625fc268f98 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:10.490316 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb1b51-e322-4769-9404-8625fc268f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:10.490768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb1b51-e322-4769-9404-8625fc268f98 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:10.491948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb1b51-e322-4769-9404-8625fc268f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:10.495456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb1b51-e322-4769-9404-8625fc268f98 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:10.550952 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 961/1918] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:10 2022] POST /identity/v3/users => generated 293 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:10.557368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731af04e-0591-4fb7-8b38-640a2776a456 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:10.592636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731af04e-0591-4fb7-8b38-640a2776a456 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:10.600140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab60842-ad68-46f6-a6fc-c3924b1189a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:10.600826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab60842-ad68-46f6-a6fc-c3924b1189a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:10.601305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab60842-ad68-46f6-a6fc-c3924b1189a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:10.602275 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab60842-ad68-46f6-a6fc-c3924b1189a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:10.603461 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab60842-ad68-46f6-a6fc-c3924b1189a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:10.606253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab60842-ad68-46f6-a6fc-c3924b1189a8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:10.634844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 958/1919] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:24:10.642406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb1b51-e322-4769-9404-8625fc268f98 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:10.673448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb1b51-e322-4769-9404-8625fc268f98 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:10.684708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24da02-2b0c-458e-aa0d-2c715885148f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:10.692341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24da02-2b0c-458e-aa0d-2c715885148f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:10.692341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24da02-2b0c-458e-aa0d-2c715885148f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:10.698086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24da02-2b0c-458e-aa0d-2c715885148f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:10.703552 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24da02-2b0c-458e-aa0d-2c715885148f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:10.715064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24da02-2b0c-458e-aa0d-2c715885148f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:10.746726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 962/1920] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:10.755812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab60842-ad68-46f6-a6fc-c3924b1189a8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:10.783869 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab60842-ad68-46f6-a6fc-c3924b1189a8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:10.815337 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df09f3-0f8a-4479-9811-b08b3268c8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:10.816507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df09f3-0f8a-4479-9811-b08b3268c8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:10.817008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df09f3-0f8a-4479-9811-b08b3268c8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:10.817492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6df09f3-0f8a-4479-9811-b08b3268c8ba admin admin] PATH_INFO: `/v3/projects/64e17b9f5890416e8c15fe1011f8cfac/users/d52467940f054e798d067d7f4d0ae516/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:10.885876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df09f3-0f8a-4479-9811-b08b3268c8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e17b9f5890416e8c15fe1011f8cfac, user_id=d52467940f054e798d067d7f4d0ae516, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:10.891008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6df09f3-0f8a-4479-9811-b08b3268c8ba admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:10.951153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 959/1921] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:10 2022] PUT /identity/v3/projects/64e17b9f5890416e8c15fe1011f8cfac/users/d52467940f054e798d067d7f4d0ae516/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:10.970480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40e0dd-6f04-47de-a0b9-6e2b305dff35 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:10.971277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40e0dd-6f04-47de-a0b9-6e2b305dff35 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:10.972116 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40e0dd-6f04-47de-a0b9-6e2b305dff35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:11.071086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ce40e0dd-6f04-47de-a0b9-6e2b305dff35 None None] MFA Rules not processed for user `d52467940f054e798d067d7f4d0ae516`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:11.072598 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ce40e0dd-6f04-47de-a0b9-6e2b305dff35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:11.277393 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 963/1922] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:11.285114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df09f3-0f8a-4479-9811-b08b3268c8ba admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:11.472911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6df09f3-0f8a-4479-9811-b08b3268c8ba admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:11.512501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db7d23-8c29-44b9-9a40-df8cc569c789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:11.514740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db7d23-8c29-44b9-9a40-df8cc569c789 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:11.515559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db7d23-8c29-44b9-9a40-df8cc569c789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:11.516390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db7d23-8c29-44b9-9a40-df8cc569c789 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:11.518392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db7d23-8c29-44b9-9a40-df8cc569c789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:11.524287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db7d23-8c29-44b9-9a40-df8cc569c789 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:11.606436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 960/1923] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:11 2022] POST /identity/v3/projects => generated 385 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:24:11.611713 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40e0dd-6f04-47de-a0b9-6e2b305dff35 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:11.626404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40e0dd-6f04-47de-a0b9-6e2b305dff35 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:11.717399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c055f-ed4c-4c2d-821d-fd62113f1d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:11.718950 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c055f-ed4c-4c2d-821d-fd62113f1d9e admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:11.719236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c055f-ed4c-4c2d-821d-fd62113f1d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:11.719513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c055f-ed4c-4c2d-821d-fd62113f1d9e admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:11.720243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c055f-ed4c-4c2d-821d-fd62113f1d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:11.722287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208c055f-ed4c-4c2d-821d-fd62113f1d9e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:11.751084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 964/1924] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:11 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:24:11.755281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db7d23-8c29-44b9-9a40-df8cc569c789 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:11.768416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db7d23-8c29-44b9-9a40-df8cc569c789 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:11.775333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ba63d-32ee-4eb7-9319-b7c1e8bfe462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:11.775981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ba63d-32ee-4eb7-9319-b7c1e8bfe462 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:11.776264 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ba63d-32ee-4eb7-9319-b7c1e8bfe462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:11.776550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ba63d-32ee-4eb7-9319-b7c1e8bfe462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:11.777220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ba63d-32ee-4eb7-9319-b7c1e8bfe462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:11.779051 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101ba63d-32ee-4eb7-9319-b7c1e8bfe462 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:11.796897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 961/1925] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:11.802646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c055f-ed4c-4c2d-821d-fd62113f1d9e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:11.820868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c055f-ed4c-4c2d-821d-fd62113f1d9e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:11.830233 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40635f-efa7-411b-ba09-8d8f62c8baf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:11.831563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40635f-efa7-411b-ba09-8d8f62c8baf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:11.832127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40635f-efa7-411b-ba09-8d8f62c8baf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:11.833049 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40635f-efa7-411b-ba09-8d8f62c8baf2 admin admin] PATH_INFO: `/v3/projects/21cfff7a3a4a4ccbb172be307ae338b5/users/51e6efe7a97c439f8a18b41614518ba1/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:11.863755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40635f-efa7-411b-ba09-8d8f62c8baf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21cfff7a3a4a4ccbb172be307ae338b5, user_id=51e6efe7a97c439f8a18b41614518ba1, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:11.869281 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40635f-efa7-411b-ba09-8d8f62c8baf2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:11.876854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:11.901174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 965/1926] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:11 2022] PUT /identity/v3/projects/21cfff7a3a4a4ccbb172be307ae338b5/users/51e6efe7a97c439f8a18b41614518ba1/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:11.906308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ba63d-32ee-4eb7-9319-b7c1e8bfe462 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:11.924304 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ba63d-32ee-4eb7-9319-b7c1e8bfe462 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:11.960007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:12.013570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd04d2c-ea69-4710-89eb-3bca92f5c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:12.014259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd04d2c-ea69-4710-89eb-3bca92f5c4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:12.014714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd04d2c-ea69-4710-89eb-3bca92f5c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:12.015154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd04d2c-ea69-4710-89eb-3bca92f5c4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:12.015989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd04d2c-ea69-4710-89eb-3bca92f5c4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:12.018172 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd04d2c-ea69-4710-89eb-3bca92f5c4b5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:12.034485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 962/1927] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:12.035671 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40635f-efa7-411b-ba09-8d8f62c8baf2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:12.050576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40635f-efa7-411b-ba09-8d8f62c8baf2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:12.058198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ddaaf-6a11-4570-ac57-2561d3deeca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:12.059917 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ddaaf-6a11-4570-ac57-2561d3deeca0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:12.060461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ddaaf-6a11-4570-ac57-2561d3deeca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:12.061147 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ddaaf-6a11-4570-ac57-2561d3deeca0 admin admin] PATH_INFO: `/v3/projects/21cfff7a3a4a4ccbb172be307ae338b5/users/51e6efe7a97c439f8a18b41614518ba1/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:12.063710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ddaaf-6a11-4570-ac57-2561d3deeca0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21cfff7a3a4a4ccbb172be307ae338b5, user_id=51e6efe7a97c439f8a18b41614518ba1, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:12.068208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ddaaf-6a11-4570-ac57-2561d3deeca0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:12.085143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 966/1928] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:12 2022] PUT /identity/v3/projects/21cfff7a3a4a4ccbb172be307ae338b5/users/51e6efe7a97c439f8a18b41614518ba1/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:12.097719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f5f2d-dc23-4db8-896f-b0d6bc26ec50 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:12.098292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f5f2d-dc23-4db8-896f-b0d6bc26ec50 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:12.098530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f5f2d-dc23-4db8-896f-b0d6bc26ec50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:12.134933 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ad3f5f2d-dc23-4db8-896f-b0d6bc26ec50 None None] MFA Rules not processed for user `51e6efe7a97c439f8a18b41614518ba1`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:12.136335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ad3f5f2d-dc23-4db8-896f-b0d6bc26ec50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:12.237391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 963/1929] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:12.250412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ddaaf-6a11-4570-ac57-2561d3deeca0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:12.252977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-940ddaaf-6a11-4570-ac57-2561d3deeca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:12.341809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ddaaf-6a11-4570-ac57-2561d3deeca0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:12.348859 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700d509d-1977-4b2c-9274-7519378d86a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:12.349812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d509d-1977-4b2c-9274-7519378d86a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:12.350459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d509d-1977-4b2c-9274-7519378d86a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:12.351004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700d509d-1977-4b2c-9274-7519378d86a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:12.496852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d509d-1977-4b2c-9274-7519378d86a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:12.500435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700d509d-1977-4b2c-9274-7519378d86a5 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:12.525441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 967/1930] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:24:12.933663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f5f2d-dc23-4db8-896f-b0d6bc26ec50 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:12.935282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ad3f5f2d-dc23-4db8-896f-b0d6bc26ec50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:13.116376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f5f2d-dc23-4db8-896f-b0d6bc26ec50 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:13.132566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:13.134171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:13.134957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:13.135750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] PATH_INFO: `/v3/users/6e83ee8ae7264504899875d4a8303a83` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:13.138618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e83ee8ae7264504899875d4a8303a83)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:13.144406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:13.225990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e83ee8ae7264504899875d4a8303a83'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:13.229534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:13.237723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e83ee8ae7264504899875d4a8303a83'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:13.238681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e83ee8ae7264504899875d4a8303a83'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:13.253012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e83ee8ae7264504899875d4a8303a83'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:13.270298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e83ee8ae7264504899875d4a8303a83'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:13.278124 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 964/1931] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:12 2022] DELETE /identity/v3/users/6e83ee8ae7264504899875d4a8303a83 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:13.283850 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fae06f-6362-4bdd-8622-78336436cda9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:13.284412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fae06f-6362-4bdd-8622-78336436cda9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:13.284541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fae06f-6362-4bdd-8622-78336436cda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:13.306269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-30fae06f-6362-4bdd-8622-78336436cda9 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:13.307702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-30fae06f-6362-4bdd-8622-78336436cda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:13.422103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 968/1932] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:13.436502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:13.438354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:13.535595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ff037-fb2c-419a-9574-455a12bacae5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:13.542271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad1bd5-c9fc-4388-80e1-3bbae48ff04d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:13.543301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad1bd5-c9fc-4388-80e1-3bbae48ff04d service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:13.543791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad1bd5-c9fc-4388-80e1-3bbae48ff04d service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:13.544285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad1bd5-c9fc-4388-80e1-3bbae48ff04d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:13.561195 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad1bd5-c9fc-4388-80e1-3bbae48ff04d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:13.563997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad1bd5-c9fc-4388-80e1-3bbae48ff04d service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:13.582204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 965/1933] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:24:14.002887 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fae06f-6362-4bdd-8622-78336436cda9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:14.006151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-30fae06f-6362-4bdd-8622-78336436cda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:14.036886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fae06f-6362-4bdd-8622-78336436cda9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:14.042710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:14.044281 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:14.044891 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:14.045232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] PATH_INFO: `/v3/projects/509e582db536430191010d80d2d1f10b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:14.047783 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=509e582db536430191010d80d2d1f10b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:14.050690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:14.110856 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:14.133240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '509e582db536430191010d80d2d1f10b', 'request_id': 'req-f62ce380-74a2-42b2-bc63-b521d7d23bf4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:14.152008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '509e582db536430191010d80d2d1f10b', 'request_id': 'req-f62ce380-74a2-42b2-bc63-b521d7d23bf4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:14.171438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '509e582db536430191010d80d2d1f10b', 'request_id': 'req-f62ce380-74a2-42b2-bc63-b521d7d23bf4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:14.208045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] The token cache is being invalidate because project 509e582db536430191010d80d2d1f10b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:14.208861 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:14.210308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:14.213233 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 969/1934] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:14 2022] DELETE /identity/v3/projects/509e582db536430191010d80d2d1f10b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:14.218964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad1bd5-c9fc-4388-80e1-3bbae48ff04d service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:14.246166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad1bd5-c9fc-4388-80e1-3bbae48ff04d service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:14.348817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f0883a-44d7-40b1-9239-6035bf892711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:14.349476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0883a-44d7-40b1-9239-6035bf892711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:14.349964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0883a-44d7-40b1-9239-6035bf892711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:14.350279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f0883a-44d7-40b1-9239-6035bf892711 admin admin] PATH_INFO: `/v3/users/d0ecc427cdf149cdb6d6b5ccf4fec156` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:14.352404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0883a-44d7-40b1-9239-6035bf892711 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0ecc427cdf149cdb6d6b5ccf4fec156)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:14.354600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f0883a-44d7-40b1-9239-6035bf892711 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:14.419619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-04f0883a-44d7-40b1-9239-6035bf892711 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0ecc427cdf149cdb6d6b5ccf4fec156'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:14.431024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-04f0883a-44d7-40b1-9239-6035bf892711 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0ecc427cdf149cdb6d6b5ccf4fec156'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:14.431867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-04f0883a-44d7-40b1-9239-6035bf892711 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0ecc427cdf149cdb6d6b5ccf4fec156'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:14.444922 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-04f0883a-44d7-40b1-9239-6035bf892711 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0ecc427cdf149cdb6d6b5ccf4fec156'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:14.457360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:14.469345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-04f0883a-44d7-40b1-9239-6035bf892711 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0ecc427cdf149cdb6d6b5ccf4fec156'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:14.479456 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 966/1935] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:14 2022] DELETE /identity/v3/users/d0ecc427cdf149cdb6d6b5ccf4fec156 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:14.957131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:14.959646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:14.989079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ce380-74a2-42b2-bc63-b521d7d23bf4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:15.069840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:15.071109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:15.071434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:15.071663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d admin admin] PATH_INFO: `/v3/projects/58161c0c02b243fc98a9b33f269f1b64` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:15.072878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d admin admin] RBAC: Authorizing `identity:delete_project(project_id=58161c0c02b243fc98a9b33f269f1b64)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:15.074980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:15.145071 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58161c0c02b243fc98a9b33f269f1b64', 'request_id': 'req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:15.163776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58161c0c02b243fc98a9b33f269f1b64', 'request_id': 'req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:15.175908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58161c0c02b243fc98a9b33f269f1b64', 'request_id': 'req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:15.202819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d admin admin] The token cache is being invalidate because project 58161c0c02b243fc98a9b33f269f1b64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:15.203967 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:15.205816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-81f24661-2c5b-4ba2-b68d-7c38c2fff50d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:15.208689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 970/1936] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:14 2022] DELETE /identity/v3/projects/58161c0c02b243fc98a9b33f269f1b64 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:15.216422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad508b79-e3aa-48e1-986e-e9392953a204 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:15.217888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad508b79-e3aa-48e1-986e-e9392953a204 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:15.218749 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad508b79-e3aa-48e1-986e-e9392953a204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:15.258681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ad508b79-e3aa-48e1-986e-e9392953a204 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:15.263718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ad508b79-e3aa-48e1-986e-e9392953a204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:15.373349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 967/1937] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:15.386455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b901b2-7632-4a3f-833d-47db0f8bebfc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:15.386930 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b901b2-7632-4a3f-833d-47db0f8bebfc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:15.387406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b901b2-7632-4a3f-833d-47db0f8bebfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:15.424734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-73b901b2-7632-4a3f-833d-47db0f8bebfc None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:15.427607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-73b901b2-7632-4a3f-833d-47db0f8bebfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:15.452153 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 971/1938] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:15.452507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad508b79-e3aa-48e1-986e-e9392953a204 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:15.466226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad508b79-e3aa-48e1-986e-e9392953a204 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:15.477186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18405472-e869-4bd0-aacd-b86c95f831cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:15.478143 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18405472-e869-4bd0-aacd-b86c95f831cf admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:15.478432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18405472-e869-4bd0-aacd-b86c95f831cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:15.478861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18405472-e869-4bd0-aacd-b86c95f831cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:15.480302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18405472-e869-4bd0-aacd-b86c95f831cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:24:15.483012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18405472-e869-4bd0-aacd-b86c95f831cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:15.501214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18405472-e869-4bd0-aacd-b86c95f831cf admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:15.553673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 968/1939] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:24:15.555694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b901b2-7632-4a3f-833d-47db0f8bebfc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:15.596857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b901b2-7632-4a3f-833d-47db0f8bebfc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:15.635315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ee300-e976-4f06-a722-848038ab2cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:15.636302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ee300-e976-4f06-a722-848038ab2cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:15.636745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ee300-e976-4f06-a722-848038ab2cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:15.637192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ee300-e976-4f06-a722-848038ab2cc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:15.650728 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ee300-e976-4f06-a722-848038ab2cc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:15.659768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ee300-e976-4f06-a722-848038ab2cc3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:15.746113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 972/1940] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:15 2022] POST /identity/v3/projects => generated 369 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:15.751126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18405472-e869-4bd0-aacd-b86c95f831cf admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:15.775313 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18405472-e869-4bd0-aacd-b86c95f831cf admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:15.789578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ee300-e976-4f06-a722-848038ab2cc3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:15.792384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-962ee300-e976-4f06-a722-848038ab2cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:15.874084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:15.953510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bb93f-66aa-47ae-a464-1b5e85cfa2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:15.954236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bb93f-66aa-47ae-a464-1b5e85cfa2ad admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:15.954703 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bb93f-66aa-47ae-a464-1b5e85cfa2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:15.955057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bb93f-66aa-47ae-a464-1b5e85cfa2ad admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:15.956030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bb93f-66aa-47ae-a464-1b5e85cfa2ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:15.958734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bb93f-66aa-47ae-a464-1b5e85cfa2ad admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:15.978242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ee300-e976-4f06-a722-848038ab2cc3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:15.993346 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 969/1941] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:15 2022] POST /identity/v3/users => generated 285 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:15.998415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf16e95-5df5-4179-878b-230b6ca2ccd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:15.999397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bb93f-66aa-47ae-a464-1b5e85cfa2ad admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:16.003303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf16e95-5df5-4179-878b-230b6ca2ccd2 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:16.004161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf16e95-5df5-4179-878b-230b6ca2ccd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:16.005051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf16e95-5df5-4179-878b-230b6ca2ccd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:16.009917 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2cf16e95-5df5-4179-878b-230b6ca2ccd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:16.027970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bb93f-66aa-47ae-a464-1b5e85cfa2ad admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:16.036077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53384ac3-2830-4b66-b8f1-c8e02474f528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:16.036714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53384ac3-2830-4b66-b8f1-c8e02474f528 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:16.036995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53384ac3-2830-4b66-b8f1-c8e02474f528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:16.037271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53384ac3-2830-4b66-b8f1-c8e02474f528 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:16.037938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53384ac3-2830-4b66-b8f1-c8e02474f528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:16.039831 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53384ac3-2830-4b66-b8f1-c8e02474f528 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:16.059702 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 970/1942] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:16.061425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53384ac3-2830-4b66-b8f1-c8e02474f528 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:16.077551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53384ac3-2830-4b66-b8f1-c8e02474f528 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:16.084116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2f61c-f213-4f52-bd15-a5b65f608e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:16.085039 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2f61c-f213-4f52-bd15-a5b65f608e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:16.085498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2f61c-f213-4f52-bd15-a5b65f608e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:16.085952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f2f61c-f213-4f52-bd15-a5b65f608e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:16.086654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2f61c-f213-4f52-bd15-a5b65f608e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:16.088760 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f2f61c-f213-4f52-bd15-a5b65f608e43 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:16.112071 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 971/1943] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:16.118083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2f61c-f213-4f52-bd15-a5b65f608e43 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:16.130287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf16e95-5df5-4179-878b-230b6ca2ccd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:16.131907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf16e95-5df5-4179-878b-230b6ca2ccd2 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:16.137365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f2f61c-f213-4f52-bd15-a5b65f608e43 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:16.145402 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b08f90-496c-4d32-a9d6-2b5c7edbe94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:16.146150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b08f90-496c-4d32-a9d6-2b5c7edbe94c admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:16.146449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b08f90-496c-4d32-a9d6-2b5c7edbe94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:16.146778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b08f90-496c-4d32-a9d6-2b5c7edbe94c admin admin] PATH_INFO: `/v3/projects/e6e4fd4426014a008600a0531f49ad2c/users/faed570558bc454cb00229f77d3fce80/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:16.156136 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 973/1944] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:16.180669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b08f90-496c-4d32-a9d6-2b5c7edbe94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e4fd4426014a008600a0531f49ad2c, user_id=faed570558bc454cb00229f77d3fce80, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:16.180669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b08f90-496c-4d32-a9d6-2b5c7edbe94c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:16.219887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 972/1945] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:16 2022] PUT /identity/v3/projects/e6e4fd4426014a008600a0531f49ad2c/users/faed570558bc454cb00229f77d3fce80/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:16.220466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acfebb-e6fb-499f-97bd-dbee7c4d0260 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:16.220763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acfebb-e6fb-499f-97bd-dbee7c4d0260 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:16.221044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52acfebb-e6fb-499f-97bd-dbee7c4d0260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:16.274844 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-52acfebb-e6fb-499f-97bd-dbee7c4d0260 None None] MFA Rules not processed for user `faed570558bc454cb00229f77d3fce80`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:16.276493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-52acfebb-e6fb-499f-97bd-dbee7c4d0260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:16.388379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 974/1946] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:16.393843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b08f90-496c-4d32-a9d6-2b5c7edbe94c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:16.422406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:16.490980 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b08f90-496c-4d32-a9d6-2b5c7edbe94c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:16.495573 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebe1f4-97b7-417f-86fa-aa90f2fda3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:16.496304 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebe1f4-97b7-417f-86fa-aa90f2fda3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:16.496768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebe1f4-97b7-417f-86fa-aa90f2fda3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:16.496924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cebe1f4-97b7-417f-86fa-aa90f2fda3f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:16.497565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cebe1f4-97b7-417f-86fa-aa90f2fda3f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:16.499735 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cebe1f4-97b7-417f-86fa-aa90f2fda3f5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:16.551956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 973/1947] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:16 2022] POST /identity/v3/projects => generated 369 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:16.556411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acfebb-e6fb-499f-97bd-dbee7c4d0260 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:16.569164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52acfebb-e6fb-499f-97bd-dbee7c4d0260 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:16.698743 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec612e32-3408-41ab-a9a7-1e25e4f6f8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:16.700922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec612e32-3408-41ab-a9a7-1e25e4f6f8bb admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:16.702341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec612e32-3408-41ab-a9a7-1e25e4f6f8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:16.702880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec612e32-3408-41ab-a9a7-1e25e4f6f8bb admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:16.704863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec612e32-3408-41ab-a9a7-1e25e4f6f8bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:16.712115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec612e32-3408-41ab-a9a7-1e25e4f6f8bb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:16.733309 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:16.758644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 975/1948] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:16 2022] POST /identity/v3/users => generated 291 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:24:16.765505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebe1f4-97b7-417f-86fa-aa90f2fda3f5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:16.788408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cebe1f4-97b7-417f-86fa-aa90f2fda3f5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:16.801147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41f3f9-be5d-42c6-aa14-bb8f0ac0b452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:16.802896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41f3f9-be5d-42c6-aa14-bb8f0ac0b452 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:16.803649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41f3f9-be5d-42c6-aa14-bb8f0ac0b452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:16.805361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41f3f9-be5d-42c6-aa14-bb8f0ac0b452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:16.807442 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41f3f9-be5d-42c6-aa14-bb8f0ac0b452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:16.812771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41f3f9-be5d-42c6-aa14-bb8f0ac0b452 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:16.849801 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 974/1949] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:24:16.852394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec612e32-3408-41ab-a9a7-1e25e4f6f8bb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:16.881982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec612e32-3408-41ab-a9a7-1e25e4f6f8bb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:16.922972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630474e-ed19-46ea-92b3-355b00b268f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:16.924302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630474e-ed19-46ea-92b3-355b00b268f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:16.924667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630474e-ed19-46ea-92b3-355b00b268f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:16.925185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630474e-ed19-46ea-92b3-355b00b268f7 admin admin] PATH_INFO: `/v3/projects/ed9b938db36b48208053fc473a1557dc/users/1e1c121675894839bd22db67fbc66ba7/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:17.022224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630474e-ed19-46ea-92b3-355b00b268f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed9b938db36b48208053fc473a1557dc, user_id=1e1c121675894839bd22db67fbc66ba7, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:17.028053 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5630474e-ed19-46ea-92b3-355b00b268f7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:17.076452 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 976/1950] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:16 2022] PUT /identity/v3/projects/ed9b938db36b48208053fc473a1557dc/users/1e1c121675894839bd22db67fbc66ba7/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:17.084260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41f3f9-be5d-42c6-aa14-bb8f0ac0b452 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:17.106399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41f3f9-be5d-42c6-aa14-bb8f0ac0b452 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:17.279292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630474e-ed19-46ea-92b3-355b00b268f7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:17.288364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397a74d-5a3c-41be-9bb2-c50e4d229888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:17.289590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397a74d-5a3c-41be-9bb2-c50e4d229888 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:17.289873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397a74d-5a3c-41be-9bb2-c50e4d229888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:17.290185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5397a74d-5a3c-41be-9bb2-c50e4d229888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:17.290898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397a74d-5a3c-41be-9bb2-c50e4d229888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:17.292692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5397a74d-5a3c-41be-9bb2-c50e4d229888 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:17.310014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 975/1951] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:17.311419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397a74d-5a3c-41be-9bb2-c50e4d229888 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:17.324249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5397a74d-5a3c-41be-9bb2-c50e4d229888 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:17.330071 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac3467-5fc6-4f69-91fc-799ad12b7b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:17.330752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac3467-5fc6-4f69-91fc-799ad12b7b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:17.331105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac3467-5fc6-4f69-91fc-799ad12b7b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:17.331451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac3467-5fc6-4f69-91fc-799ad12b7b48 admin admin] PATH_INFO: `/v3/projects/ed9b938db36b48208053fc473a1557dc/users/1e1c121675894839bd22db67fbc66ba7/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:17.334203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac3467-5fc6-4f69-91fc-799ad12b7b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed9b938db36b48208053fc473a1557dc, user_id=1e1c121675894839bd22db67fbc66ba7, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:17.336733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aac3467-5fc6-4f69-91fc-799ad12b7b48 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:17.352979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 976/1952] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:17 2022] PUT /identity/v3/projects/ed9b938db36b48208053fc473a1557dc/users/1e1c121675894839bd22db67fbc66ba7/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:17.371444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d990f1-63ae-4a2b-afc8-cb855f2871be None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:17.371444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d990f1-63ae-4a2b-afc8-cb855f2871be None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:17.371444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d990f1-63ae-4a2b-afc8-cb855f2871be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:17.384345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630474e-ed19-46ea-92b3-355b00b268f7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:17.411408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a9d990f1-63ae-4a2b-afc8-cb855f2871be None None] MFA Rules not processed for user `1e1c121675894839bd22db67fbc66ba7`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:17.412713 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a9d990f1-63ae-4a2b-afc8-cb855f2871be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:17.463338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5cb4e-73e4-4b93-a6da-9dee5500601b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:17.463962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5cb4e-73e4-4b93-a6da-9dee5500601b service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:17.464239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5cb4e-73e4-4b93-a6da-9dee5500601b service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:17.464533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae5cb4e-73e4-4b93-a6da-9dee5500601b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:17.466431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fae5cb4e-73e4-4b93-a6da-9dee5500601b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:17.531620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 977/1953] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:17.551036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d990f1-63ae-4a2b-afc8-cb855f2871be None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:17.552998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a9d990f1-63ae-4a2b-afc8-cb855f2871be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:17.558756 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5cb4e-73e4-4b93-a6da-9dee5500601b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:17.560688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae5cb4e-73e4-4b93-a6da-9dee5500601b service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:17.564669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:17.573997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d990f1-63ae-4a2b-afc8-cb855f2871be None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:17.574325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 977/1954] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:17 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:24:17.579914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5243d7-01b5-4465-9030-329e3c56c8f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:17.581181 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5243d7-01b5-4465-9030-329e3c56c8f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:17.581466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5243d7-01b5-4465-9030-329e3c56c8f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:17.582087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5243d7-01b5-4465-9030-329e3c56c8f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:17.596382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5243d7-01b5-4465-9030-329e3c56c8f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:17.599369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5243d7-01b5-4465-9030-329e3c56c8f7 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:17.619407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 978/1955] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:24:17.690547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5cb4e-73e4-4b93-a6da-9dee5500601b service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:17.703411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae5cb4e-73e4-4b93-a6da-9dee5500601b service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:17.708118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c404b-277a-41f7-88d2-a755a229365d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:17.709294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c404b-277a-41f7-88d2-a755a229365d service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:17.709808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c404b-277a-41f7-88d2-a755a229365d service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:17.710549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c404b-277a-41f7-88d2-a755a229365d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:17.766567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:17.806029 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c404b-277a-41f7-88d2-a755a229365d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:17.808152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c404b-277a-41f7-88d2-a755a229365d service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:17.819849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 978/1956] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:21.819423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5243d7-01b5-4465-9030-329e3c56c8f7 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:21.951351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c404b-277a-41f7-88d2-a755a229365d service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:21.954886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bd5c404b-277a-41f7-88d2-a755a229365d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:22.007559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5243d7-01b5-4465-9030-329e3c56c8f7 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:22.029086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29aff2b-d656-4f6e-820b-5a2fc167330c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:22.029854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aff2b-d656-4f6e-820b-5a2fc167330c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:22.030410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aff2b-d656-4f6e-820b-5a2fc167330c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:22.030776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29aff2b-d656-4f6e-820b-5a2fc167330c admin admin] PATH_INFO: `/v3/users/faed570558bc454cb00229f77d3fce80` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:22.034436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29aff2b-d656-4f6e-820b-5a2fc167330c admin admin] RBAC: Authorizing `identity:delete_user(user_id=faed570558bc454cb00229f77d3fce80)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:22.038813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29aff2b-d656-4f6e-820b-5a2fc167330c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:22.098665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c404b-277a-41f7-88d2-a755a229365d service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:22.105090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60726b3-7301-45f5-af83-39f1e6ad28e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:22.106112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60726b3-7301-45f5-af83-39f1e6ad28e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:22.106615 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60726b3-7301-45f5-af83-39f1e6ad28e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:22.107073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60726b3-7301-45f5-af83-39f1e6ad28e5 admin admin] PATH_INFO: `/v3/users/24e4e8f827e34b44a457a5b4218b22f1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:22.108580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60726b3-7301-45f5-af83-39f1e6ad28e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e4e8f827e34b44a457a5b4218b22f1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:22.111271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60726b3-7301-45f5-af83-39f1e6ad28e5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:22.173598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c29aff2b-d656-4f6e-820b-5a2fc167330c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faed570558bc454cb00229f77d3fce80'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:22.191321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c29aff2b-d656-4f6e-820b-5a2fc167330c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faed570558bc454cb00229f77d3fce80'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:22.193114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c29aff2b-d656-4f6e-820b-5a2fc167330c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faed570558bc454cb00229f77d3fce80'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:22.207844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c29aff2b-d656-4f6e-820b-5a2fc167330c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faed570558bc454cb00229f77d3fce80'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:22.217288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e60726b3-7301-45f5-af83-39f1e6ad28e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e4e8f827e34b44a457a5b4218b22f1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:22.230404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e60726b3-7301-45f5-af83-39f1e6ad28e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e4e8f827e34b44a457a5b4218b22f1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:22.245636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c29aff2b-d656-4f6e-820b-5a2fc167330c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faed570558bc454cb00229f77d3fce80'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:22.257869 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e60726b3-7301-45f5-af83-39f1e6ad28e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e4e8f827e34b44a457a5b4218b22f1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:22.258730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e60726b3-7301-45f5-af83-39f1e6ad28e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e4e8f827e34b44a457a5b4218b22f1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:22.260139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 979/1957] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:21 2022] DELETE /identity/v3/users/faed570558bc454cb00229f77d3fce80 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:22.268974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e60726b3-7301-45f5-af83-39f1e6ad28e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e4e8f827e34b44a457a5b4218b22f1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:22.269284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e9b53-a4f1-475b-8763-f9606117a6be None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:22.269478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e9b53-a4f1-475b-8763-f9606117a6be None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:22.269955 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e9b53-a4f1-475b-8763-f9606117a6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:22.282103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 979/1958] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:21 2022] DELETE /identity/v3/users/24e4e8f827e34b44a457a5b4218b22f1 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:22.284293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c751c7-35d7-4986-95f3-b8c24e190fb4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:22.284583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c751c7-35d7-4986-95f3-b8c24e190fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:22.284863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c751c7-35d7-4986-95f3-b8c24e190fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:22.315412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-55c751c7-35d7-4986-95f3-b8c24e190fb4 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:22.316273 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-501e9b53-a4f1-475b-8763-f9606117a6be None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:22.316681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-55c751c7-35d7-4986-95f3-b8c24e190fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:22.318503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-501e9b53-a4f1-475b-8763-f9606117a6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:22.437669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 980/1959] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:22.451424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:22.455710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e9b53-a4f1-475b-8763-f9606117a6be None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:22.461971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-501e9b53-a4f1-475b-8763-f9606117a6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:22.478288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 980/1960] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:22.490593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c751c7-35d7-4986-95f3-b8c24e190fb4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:22.494572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:22.496818 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-55c751c7-35d7-4986-95f3-b8c24e190fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:22.577483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e9b53-a4f1-475b-8763-f9606117a6be None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:22.585210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd06987-faae-45a7-a814-4690fe0fd957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:22.585210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd06987-faae-45a7-a814-4690fe0fd957 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:22.585210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd06987-faae-45a7-a814-4690fe0fd957 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:22.585210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd06987-faae-45a7-a814-4690fe0fd957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:22.595765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd06987-faae-45a7-a814-4690fe0fd957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:22.597231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd06987-faae-45a7-a814-4690fe0fd957 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:22.610788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 981/1961] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:24:22.611076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c751c7-35d7-4986-95f3-b8c24e190fb4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:22.631398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab822215-4670-42f4-9260-98558c7064ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:22.631398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab822215-4670-42f4-9260-98558c7064ac service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:22.631398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab822215-4670-42f4-9260-98558c7064ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:22.631398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab822215-4670-42f4-9260-98558c7064ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:22.644978 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab822215-4670-42f4-9260-98558c7064ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:22.648201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab822215-4670-42f4-9260-98558c7064ac service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:22.669980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 981/1962] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:23.550701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd06987-faae-45a7-a814-4690fe0fd957 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:23.554276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5fd06987-faae-45a7-a814-4690fe0fd957 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:23.609796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd06987-faae-45a7-a814-4690fe0fd957 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:23.621749 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:23.623343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:23.624076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:23.624777 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] PATH_INFO: `/v3/projects/20c1349283a747ac8582022a84887e04` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:23.627515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20c1349283a747ac8582022a84887e04)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:23.632724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:23.747434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20c1349283a747ac8582022a84887e04', 'request_id': 'req-69b8a779-2326-44c5-b9fd-67ab51179762'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:23.756249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20c1349283a747ac8582022a84887e04', 'request_id': 'req-69b8a779-2326-44c5-b9fd-67ab51179762'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:23.765271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20c1349283a747ac8582022a84887e04', 'request_id': 'req-69b8a779-2326-44c5-b9fd-67ab51179762'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:23.787043 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab822215-4670-42f4-9260-98558c7064ac service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:23.789100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ab822215-4670-42f4-9260-98558c7064ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:23.793786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] The token cache is being invalidate because project 20c1349283a747ac8582022a84887e04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:23.794537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:23.795685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:23.802451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 982/1963] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:23 2022] DELETE /identity/v3/projects/20c1349283a747ac8582022a84887e04 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:23.805684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:23.808405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:23.818432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab822215-4670-42f4-9260-98558c7064ac service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:23.919481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:23.920568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:23.921165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:23.921764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] PATH_INFO: `/v3/projects/e6e4fd4426014a008600a0531f49ad2c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:23.924175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6e4fd4426014a008600a0531f49ad2c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:23.928427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:23.929868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8a779-2326-44c5-b9fd-67ab51179762 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:23.936615 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:23.937741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:23.938270 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:23.938631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] PATH_INFO: `/v3/users/d52467940f054e798d067d7f4d0ae516` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:23.940398 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52467940f054e798d067d7f4d0ae516)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:23.942823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:23.998234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:24.016165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52467940f054e798d067d7f4d0ae516'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.034015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52467940f054e798d067d7f4d0ae516'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.034956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52467940f054e798d067d7f4d0ae516'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.036899 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6e4fd4426014a008600a0531f49ad2c', 'request_id': 'req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.052495 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52467940f054e798d067d7f4d0ae516'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.069977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6e4fd4426014a008600a0531f49ad2c', 'request_id': 'req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.072974 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52467940f054e798d067d7f4d0ae516'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.080565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6e4fd4426014a008600a0531f49ad2c', 'request_id': 'req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.084789 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 983/1964] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:23 2022] DELETE /identity/v3/users/d52467940f054e798d067d7f4d0ae516 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:24.097003 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] The token cache is being invalidate because project e6e4fd4426014a008600a0531f49ad2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:24.097385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.098266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.099979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 982/1965] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:23 2022] DELETE /identity/v3/projects/e6e4fd4426014a008600a0531f49ad2c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:24.107350 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:24.109597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:24.230805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665118f3-1275-43db-ade7-6cbe80ba7c56 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:24.251546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:24.253676 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:24.254584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:24.256199 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] PATH_INFO: `/v3/users/1e1c121675894839bd22db67fbc66ba7` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:24.260248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e1c121675894839bd22db67fbc66ba7)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:24.264053 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:24.405040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e1c121675894839bd22db67fbc66ba7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.431484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e1c121675894839bd22db67fbc66ba7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.432846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e1c121675894839bd22db67fbc66ba7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.464923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e1c121675894839bd22db67fbc66ba7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.511306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e1c121675894839bd22db67fbc66ba7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:24.528503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 984/1966] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:24 2022] DELETE /identity/v3/users/1e1c121675894839bd22db67fbc66ba7 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:24.826996 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:24.829467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:24.914742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:24.967517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be68f4f-712f-4c20-9b11-fc9cacdc0c71 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:25.048644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:25.049316 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:25.049619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:25.049903 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] PATH_INFO: `/v3/projects/64e17b9f5890416e8c15fe1011f8cfac` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:25.051235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e17b9f5890416e8c15fe1011f8cfac)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:25.054522 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:25.136925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e17b9f5890416e8c15fe1011f8cfac', 'request_id': 'req-22992507-f96d-4899-81c8-4275ac0cb25e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.152640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e17b9f5890416e8c15fe1011f8cfac', 'request_id': 'req-22992507-f96d-4899-81c8-4275ac0cb25e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.175556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e17b9f5890416e8c15fe1011f8cfac', 'request_id': 'req-22992507-f96d-4899-81c8-4275ac0cb25e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.191953 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] The token cache is being invalidate because project 64e17b9f5890416e8c15fe1011f8cfac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:25.192479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.193875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.196278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 983/1967] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:24 2022] DELETE /identity/v3/projects/64e17b9f5890416e8c15fe1011f8cfac => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:25.202716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:25.207566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:25.271536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994948f4-af30-4686-b4a5-60ccc43792d2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:25.361240 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:25.421527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71a5bf-a8d3-4a64-a395-322d1ace4b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:25.423269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71a5bf-a8d3-4a64-a395-322d1ace4b3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:25.424401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71a5bf-a8d3-4a64-a395-322d1ace4b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:25.425493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71a5bf-a8d3-4a64-a395-322d1ace4b3f admin admin] PATH_INFO: `/v3/users/51e6efe7a97c439f8a18b41614518ba1` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:25.431141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71a5bf-a8d3-4a64-a395-322d1ace4b3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=51e6efe7a97c439f8a18b41614518ba1)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:25.436217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71a5bf-a8d3-4a64-a395-322d1ace4b3f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:25.529774 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:25.532297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:25.579185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22992507-f96d-4899-81c8-4275ac0cb25e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:25.588088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d07d9-340a-4b95-b43a-205141e751fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:25.588926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d07d9-340a-4b95-b43a-205141e751fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:25.589371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d07d9-340a-4b95-b43a-205141e751fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:25.591109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d07d9-340a-4b95-b43a-205141e751fe admin admin] PATH_INFO: `/v3/projects/ed9b938db36b48208053fc473a1557dc` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:25.592778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d07d9-340a-4b95-b43a-205141e751fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed9b938db36b48208053fc473a1557dc)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:25.595943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d07d9-340a-4b95-b43a-205141e751fe admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:25.633008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1f71a5bf-a8d3-4a64-a395-322d1ace4b3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51e6efe7a97c439f8a18b41614518ba1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.644043 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1f71a5bf-a8d3-4a64-a395-322d1ace4b3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51e6efe7a97c439f8a18b41614518ba1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.644699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1f71a5bf-a8d3-4a64-a395-322d1ace4b3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51e6efe7a97c439f8a18b41614518ba1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.663526 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1f71a5bf-a8d3-4a64-a395-322d1ace4b3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51e6efe7a97c439f8a18b41614518ba1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.701736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4d8d07d9-340a-4b95-b43a-205141e751fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed9b938db36b48208053fc473a1557dc', 'request_id': 'req-4d8d07d9-340a-4b95-b43a-205141e751fe'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.701982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1f71a5bf-a8d3-4a64-a395-322d1ace4b3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51e6efe7a97c439f8a18b41614518ba1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.718184 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 985/1968] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:25 2022] DELETE /identity/v3/users/51e6efe7a97c439f8a18b41614518ba1 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:25.724882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4d8d07d9-340a-4b95-b43a-205141e751fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed9b938db36b48208053fc473a1557dc', 'request_id': 'req-4d8d07d9-340a-4b95-b43a-205141e751fe'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.736142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:25.751826 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4d8d07d9-340a-4b95-b43a-205141e751fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed9b938db36b48208053fc473a1557dc', 'request_id': 'req-4d8d07d9-340a-4b95-b43a-205141e751fe'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.796185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4d8d07d9-340a-4b95-b43a-205141e751fe admin admin] The token cache is being invalidate because project ed9b938db36b48208053fc473a1557dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:25.796514 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4d8d07d9-340a-4b95-b43a-205141e751fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.799223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4d8d07d9-340a-4b95-b43a-205141e751fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:25.801263 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 984/1969] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:25 2022] DELETE /identity/v3/projects/ed9b938db36b48208053fc473a1557dc => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:25.807839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a921c-4731-47c0-90e7-97ac61b79f46 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:25.808929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a921c-4731-47c0-90e7-97ac61b79f46 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:25.809584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a921c-4731-47c0-90e7-97ac61b79f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:25.890037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-158a921c-4731-47c0-90e7-97ac61b79f46 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:25.892918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-158a921c-4731-47c0-90e7-97ac61b79f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:26.069960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 986/1970] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:26.078927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209fd08-01c1-4fff-89f8-7fadad793ac5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.079722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209fd08-01c1-4fff-89f8-7fadad793ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.079722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209fd08-01c1-4fff-89f8-7fadad793ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:26.105350 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3209fd08-01c1-4fff-89f8-7fadad793ac5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:26.106788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3209fd08-01c1-4fff-89f8-7fadad793ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:26.120926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 985/1971] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:26.125004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a921c-4731-47c0-90e7-97ac61b79f46 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:26.150796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158a921c-4731-47c0-90e7-97ac61b79f46 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:26.159767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad22e58-7893-43f9-9b54-d70793b66986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:26.160366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad22e58-7893-43f9-9b54-d70793b66986 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.160882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad22e58-7893-43f9-9b54-d70793b66986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.161322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad22e58-7893-43f9-9b54-d70793b66986 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:26.162589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad22e58-7893-43f9-9b54-d70793b66986 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:24:26.163143 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad22e58-7893-43f9-9b54-d70793b66986 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:26.165778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad22e58-7893-43f9-9b54-d70793b66986 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:26.193525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 987/1972] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:24:26.196033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209fd08-01c1-4fff-89f8-7fadad793ac5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:26.208563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209fd08-01c1-4fff-89f8-7fadad793ac5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:26.227144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc9922-c468-4ed9-9bea-6b9882a035ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:26.228869 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc9922-c468-4ed9-9bea-6b9882a035ca admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.229259 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc9922-c468-4ed9-9bea-6b9882a035ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.229643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc9922-c468-4ed9-9bea-6b9882a035ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:26.230889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc9922-c468-4ed9-9bea-6b9882a035ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:26.233484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc9922-c468-4ed9-9bea-6b9882a035ca admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:26.302139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 986/1973] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:26 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:26.306712 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad22e58-7893-43f9-9b54-d70793b66986 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:26.325421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad22e58-7893-43f9-9b54-d70793b66986 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:26.440780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:26.501039 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535bf2a-012c-4dd3-a034-64f5adce413b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:26.503498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535bf2a-012c-4dd3-a034-64f5adce413b admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.503781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535bf2a-012c-4dd3-a034-64f5adce413b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.504058 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b535bf2a-012c-4dd3-a034-64f5adce413b admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:26.504739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535bf2a-012c-4dd3-a034-64f5adce413b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:26.511628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b535bf2a-012c-4dd3-a034-64f5adce413b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:26.572783 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc9922-c468-4ed9-9bea-6b9882a035ca admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:26.575902 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-48cc9922-c468-4ed9-9bea-6b9882a035ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:26.581993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 988/1974] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:26 2022] POST /identity/v3/users => generated 294 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:26.583467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535bf2a-012c-4dd3-a034-64f5adce413b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:26.597292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b535bf2a-012c-4dd3-a034-64f5adce413b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:26.608611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f62a9-5bec-41b1-8959-0176cb882f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:26.608611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f62a9-5bec-41b1-8959-0176cb882f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.608611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f62a9-5bec-41b1-8959-0176cb882f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.608611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f62a9-5bec-41b1-8959-0176cb882f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:26.609191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f62a9-5bec-41b1-8959-0176cb882f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:26.611287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9f62a9-5bec-41b1-8959-0176cb882f0d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:26.617283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc9922-c468-4ed9-9bea-6b9882a035ca admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:26.626086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 989/1975] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:26.631224 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f62a9-5bec-41b1-8959-0176cb882f0d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:26.639245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:26.645552 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.645865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.646185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd admin admin] PATH_INFO: `/v3/projects/21cfff7a3a4a4ccbb172be307ae338b5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:26.650625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=21cfff7a3a4a4ccbb172be307ae338b5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:26.651623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9f62a9-5bec-41b1-8959-0176cb882f0d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:26.658180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:26.663311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a17e1-a410-49c9-89d7-061b03b9e5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:26.664166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a17e1-a410-49c9-89d7-061b03b9e5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.664454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a17e1-a410-49c9-89d7-061b03b9e5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.664730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a17e1-a410-49c9-89d7-061b03b9e5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:26.665622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a17e1-a410-49c9-89d7-061b03b9e5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:26.667965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a17e1-a410-49c9-89d7-061b03b9e5d0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:26.688973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 990/1976] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:26.696272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a17e1-a410-49c9-89d7-061b03b9e5d0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:26.720755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a17e1-a410-49c9-89d7-061b03b9e5d0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:26.736798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21cfff7a3a4a4ccbb172be307ae338b5', 'request_id': 'req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:26.751475 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21cfff7a3a4a4ccbb172be307ae338b5', 'request_id': 'req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:26.767468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21cfff7a3a4a4ccbb172be307ae338b5', 'request_id': 'req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:26.804384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd admin admin] The token cache is being invalidate because project 21cfff7a3a4a4ccbb172be307ae338b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:26.804934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:26.806241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d188310a-bbfd-42ac-96c5-9af1ea2fe3cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:26.808955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 987/1977] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:26 2022] DELETE /identity/v3/projects/21cfff7a3a4a4ccbb172be307ae338b5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:26.814010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87e049-1ced-4ceb-b0d8-308f9dec2426 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.814406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87e049-1ced-4ceb-b0d8-308f9dec2426 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.814775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c87e049-1ced-4ceb-b0d8-308f9dec2426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:26.818138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467607f-9951-498b-95ff-e8b1b599b9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:26.819512 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467607f-9951-498b-95ff-e8b1b599b9c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.820224 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467607f-9951-498b-95ff-e8b1b599b9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.820740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467607f-9951-498b-95ff-e8b1b599b9c7 admin admin] PATH_INFO: `/v3/projects/fc571556c1c045a99ec75bf7cbda9b91/users/9efb2299c314422b85a733f09c7ff184/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:26.836718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3c87e049-1ced-4ceb-b0d8-308f9dec2426 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:26.838140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3c87e049-1ced-4ceb-b0d8-308f9dec2426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:26.847546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 988/1978] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:26.857696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcce87-b527-4e98-991f-58c28511adc8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.858326 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcce87-b527-4e98-991f-58c28511adc8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.858694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dcce87-b527-4e98-991f-58c28511adc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:26.861417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467607f-9951-498b-95ff-e8b1b599b9c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc571556c1c045a99ec75bf7cbda9b91, user_id=9efb2299c314422b85a733f09c7ff184, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:26.864271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467607f-9951-498b-95ff-e8b1b599b9c7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:26.893197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 991/1979] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:26 2022] PUT /identity/v3/projects/fc571556c1c045a99ec75bf7cbda9b91/users/9efb2299c314422b85a733f09c7ff184/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:26.906506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b3f4e-ccad-4630-96c0-3113366ed94b None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.906506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b3f4e-ccad-4630-96c0-3113366ed94b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.906506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b3f4e-ccad-4630-96c0-3113366ed94b None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:26.906506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 992/1980] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:24:26.907126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-09dcce87-b527-4e98-991f-58c28511adc8 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:26.907126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-09dcce87-b527-4e98-991f-58c28511adc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:26.912068 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cab67-3663-4946-94bf-e6b05b87ebe0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.912391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cab67-3663-4946-94bf-e6b05b87ebe0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.912870 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33cab67-3663-4946-94bf-e6b05b87ebe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:26.950929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d33cab67-3663-4946-94bf-e6b05b87ebe0 None None] MFA Rules not processed for user `9efb2299c314422b85a733f09c7ff184`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:26.952353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d33cab67-3663-4946-94bf-e6b05b87ebe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:26.993345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 989/1981] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:26.994579 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f152392-b54d-4d86-965b-54daf04cfd5a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:26.994881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f152392-b54d-4d86-965b-54daf04cfd5a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:26.995164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f152392-b54d-4d86-965b-54daf04cfd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.039338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0f152392-b54d-4d86-965b-54daf04cfd5a None None] MFA Rules not processed for user `55642294e8ff4f779510f4646e0e30a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:27.040936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0f152392-b54d-4d86-965b-54daf04cfd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:27.101129 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 993/1982] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:27.103139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cab67-3663-4946-94bf-e6b05b87ebe0 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:27.108195 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:27.119416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33cab67-3663-4946-94bf-e6b05b87ebe0 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:27.125926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358dade-e0ea-4b0d-8ce8-83d353c9b041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:27.126920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358dade-e0ea-4b0d-8ce8-83d353c9b041 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:27.127235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358dade-e0ea-4b0d-8ce8-83d353c9b041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:27.127472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a358dade-e0ea-4b0d-8ce8-83d353c9b041 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.128373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358dade-e0ea-4b0d-8ce8-83d353c9b041 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:24:27.128790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358dade-e0ea-4b0d-8ce8-83d353c9b041 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:27.131057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a358dade-e0ea-4b0d-8ce8-83d353c9b041 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:27.151642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 994/1983] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:24:27.151642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358dade-e0ea-4b0d-8ce8-83d353c9b041 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:27.152067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 990/1984] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:24:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:27.152067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f152392-b54d-4d86-965b-54daf04cfd5a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:27.152342 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a358dade-e0ea-4b0d-8ce8-83d353c9b041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:27.152489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0f152392-b54d-4d86-965b-54daf04cfd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:27.168686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f152392-b54d-4d86-965b-54daf04cfd5a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:27.169160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a358dade-e0ea-4b0d-8ce8-83d353c9b041 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:27.173947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72154270-d025-4535-861b-d9de9b8566d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:27.174603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72154270-d025-4535-861b-d9de9b8566d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:27.174881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72154270-d025-4535-861b-d9de9b8566d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:27.175189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72154270-d025-4535-861b-d9de9b8566d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.175440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765494d-8f6b-4a98-8745-0044e9529a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:27.175881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72154270-d025-4535-861b-d9de9b8566d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:27.177171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765494d-8f6b-4a98-8745-0044e9529a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:27.177826 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72154270-d025-4535-861b-d9de9b8566d7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:27.178345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765494d-8f6b-4a98-8745-0044e9529a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:27.180784 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765494d-8f6b-4a98-8745-0044e9529a90 admin admin] PATH_INFO: `/v3/users/a4f83757d3484a58b89ec39d46effaab` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.184804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765494d-8f6b-4a98-8745-0044e9529a90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4f83757d3484a58b89ec39d46effaab)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:27.187101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765494d-8f6b-4a98-8745-0044e9529a90 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:27.189387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:27.223193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 991/1985] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:27 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:27.223193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72154270-d025-4535-861b-d9de9b8566d7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:27.259519 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4765494d-8f6b-4a98-8745-0044e9529a90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4f83757d3484a58b89ec39d46effaab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:27.269789 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4765494d-8f6b-4a98-8745-0044e9529a90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4f83757d3484a58b89ec39d46effaab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:27.270787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4765494d-8f6b-4a98-8745-0044e9529a90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4f83757d3484a58b89ec39d46effaab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:27.281105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4765494d-8f6b-4a98-8745-0044e9529a90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4f83757d3484a58b89ec39d46effaab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:27.297120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4765494d-8f6b-4a98-8745-0044e9529a90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4f83757d3484a58b89ec39d46effaab'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:27.304767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 995/1986] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:27 2022] DELETE /identity/v3/users/a4f83757d3484a58b89ec39d46effaab => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:27.306217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd6e84-282b-47de-b6c4-f239ea5701c9 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:27.306504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd6e84-282b-47de-b6c4-f239ea5701c9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:27.306776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd6e84-282b-47de-b6c4-f239ea5701c9 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.308348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 996/1987] 10.222.0.64 () {54 vars in 961 bytes} [Wed Sep 21 16:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:24:27.309249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd6e84-282b-47de-b6c4-f239ea5701c9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:27.310810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cecd6e84-282b-47de-b6c4-f239ea5701c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:27.321602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72154270-d025-4535-861b-d9de9b8566d7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:27.418379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb154aed-95ff-4caf-8e7c-9092519e81ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:27.419176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb154aed-95ff-4caf-8e7c-9092519e81ab admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:27.419459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb154aed-95ff-4caf-8e7c-9092519e81ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:27.420386 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb154aed-95ff-4caf-8e7c-9092519e81ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.421091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb154aed-95ff-4caf-8e7c-9092519e81ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:27.423590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb154aed-95ff-4caf-8e7c-9092519e81ab admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:27.426465 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd6e84-282b-47de-b6c4-f239ea5701c9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:27.432973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb35e38-4d21-46c9-ad1b-dbd5c4f78640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:27.433729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb35e38-4d21-46c9-ad1b-dbd5c4f78640 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:27.434363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb35e38-4d21-46c9-ad1b-dbd5c4f78640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:27.434894 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb35e38-4d21-46c9-ad1b-dbd5c4f78640 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.435681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb35e38-4d21-46c9-ad1b-dbd5c4f78640 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:27.438083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb35e38-4d21-46c9-ad1b-dbd5c4f78640 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:27.476748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 997/1988] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:27 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:27.478216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90944d1d-5c5d-4b46-89e6-c142902a5a2c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:27.478502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90944d1d-5c5d-4b46-89e6-c142902a5a2c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:27.478787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90944d1d-5c5d-4b46-89e6-c142902a5a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.485067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 992/1989] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:27 2022] POST /identity/v3/projects => generated 371 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:27.486328 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb154aed-95ff-4caf-8e7c-9092519e81ab admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:27.530291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-90944d1d-5c5d-4b46-89e6-c142902a5a2c None None] MFA Rules not processed for user `55642294e8ff4f779510f4646e0e30a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:27.531599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-90944d1d-5c5d-4b46-89e6-c142902a5a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:27.633418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:27.642508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 998/1990] 10.222.0.64 () {58 vars in 1058 bytes} [Wed Sep 21 16:24:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:27.643397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90944d1d-5c5d-4b46-89e6-c142902a5a2c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:27.644896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-90944d1d-5c5d-4b46-89e6-c142902a5a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:27.653506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:27.688374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb154aed-95ff-4caf-8e7c-9092519e81ab admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:27.695160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee136588-02ce-4eed-8c05-ffcc6590138d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:27.696330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee136588-02ce-4eed-8c05-ffcc6590138d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:27.696743 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee136588-02ce-4eed-8c05-ffcc6590138d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:27.697189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee136588-02ce-4eed-8c05-ffcc6590138d admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.698404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee136588-02ce-4eed-8c05-ffcc6590138d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:27.700468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee136588-02ce-4eed-8c05-ffcc6590138d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:27.713799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 993/1991] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:27.714697 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee136588-02ce-4eed-8c05-ffcc6590138d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:27.731193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee136588-02ce-4eed-8c05-ffcc6590138d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:27.738442 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d3c8d-d054-43bf-b327-3795d31a3cd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:27.739128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d3c8d-d054-43bf-b327-3795d31a3cd8 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:27.739747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d3c8d-d054-43bf-b327-3795d31a3cd8 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:27.740068 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d3c8d-d054-43bf-b327-3795d31a3cd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.747634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-110d3c8d-d054-43bf-b327-3795d31a3cd8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:27.751473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90944d1d-5c5d-4b46-89e6-c142902a5a2c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:27.756513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c121b53-a29f-4cd5-9360-b9687e2e5d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:27.757292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c121b53-a29f-4cd5-9360-b9687e2e5d59 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:27.757672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c121b53-a29f-4cd5-9360-b9687e2e5d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:27.757953 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c121b53-a29f-4cd5-9360-b9687e2e5d59 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.758749 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c121b53-a29f-4cd5-9360-b9687e2e5d59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:27.760806 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d3c8d-d054-43bf-b327-3795d31a3cd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:27.760930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c121b53-a29f-4cd5-9360-b9687e2e5d59 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:27.762533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d3c8d-d054-43bf-b327-3795d31a3cd8 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:27.773935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 994/1992] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:27.776669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d3c8d-d054-43bf-b327-3795d31a3cd8 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:27.788892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d3c8d-d054-43bf-b327-3795d31a3cd8 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:27.794972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff248e6-043a-4933-95c5-aca484687a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:27.795987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff248e6-043a-4933-95c5-aca484687a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:27.796099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff248e6-043a-4933-95c5-aca484687a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:27.796344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff248e6-043a-4933-95c5-aca484687a96 admin admin] PATH_INFO: `/v3/projects/528d667178324778bc25dd20bce66a12/users/88d3745e1dea492cb28b58809f2a421b/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.805881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 999/1993] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:27 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:27.805881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c121b53-a29f-4cd5-9360-b9687e2e5d59 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:27.809226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5c121b53-a29f-4cd5-9360-b9687e2e5d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:27.826351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c121b53-a29f-4cd5-9360-b9687e2e5d59 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:27.827839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff248e6-043a-4933-95c5-aca484687a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=528d667178324778bc25dd20bce66a12, user_id=88d3745e1dea492cb28b58809f2a421b, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:27.832785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:27.833119 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff248e6-043a-4933-95c5-aca484687a96 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:27.833294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:27.833294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:27.833294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] PATH_INFO: `/v3/projects/b94ef89f42ce4bc9920e9ac5f1c573e4` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:27.834723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b94ef89f42ce4bc9920e9ac5f1c573e4)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:27.837493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:27.859578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 995/1994] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:27 2022] PUT /identity/v3/projects/528d667178324778bc25dd20bce66a12/users/88d3745e1dea492cb28b58809f2a421b/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:27.861045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff248e6-043a-4933-95c5-aca484687a96 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:27.909375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b94ef89f42ce4bc9920e9ac5f1c573e4', 'request_id': 'req-5c1ad3eb-82f5-45c7-8545-17f038f07109'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:27.920528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b94ef89f42ce4bc9920e9ac5f1c573e4', 'request_id': 'req-5c1ad3eb-82f5-45c7-8545-17f038f07109'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:27.931088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b94ef89f42ce4bc9920e9ac5f1c573e4', 'request_id': 'req-5c1ad3eb-82f5-45c7-8545-17f038f07109'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:27.948746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff248e6-043a-4933-95c5-aca484687a96 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:27.952970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] The token cache is being invalidate because project b94ef89f42ce4bc9920e9ac5f1c573e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:27.953451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:27.954501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:27.958071 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1000/1995] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:27 2022] DELETE /identity/v3/projects/b94ef89f42ce4bc9920e9ac5f1c573e4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:27.959497 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:27.962138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:28.023837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c4249-b3c8-48a1-82e5-0e60145b531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.024523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c4249-b3c8-48a1-82e5-0e60145b531f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.024905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c4249-b3c8-48a1-82e5-0e60145b531f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.025279 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c4249-b3c8-48a1-82e5-0e60145b531f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.026039 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c4249-b3c8-48a1-82e5-0e60145b531f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.027798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c4249-b3c8-48a1-82e5-0e60145b531f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.042010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 996/1996] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:28.042010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05092ac-b5ff-4567-aaf9-080c1425d4c6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.042010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05092ac-b5ff-4567-aaf9-080c1425d4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.042010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05092ac-b5ff-4567-aaf9-080c1425d4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.048382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ad3eb-82f5-45c7-8545-17f038f07109 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.063019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7c2b0b-6674-40b6-8c05-37c7c00ec240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.064109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c2b0b-6674-40b6-8c05-37c7c00ec240 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.065789 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c2b0b-6674-40b6-8c05-37c7c00ec240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.066305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7c2b0b-6674-40b6-8c05-37c7c00ec240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.067135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c2b0b-6674-40b6-8c05-37c7c00ec240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.071560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7c2b0b-6674-40b6-8c05-37c7c00ec240 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.072009 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f05092ac-b5ff-4567-aaf9-080c1425d4c6 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:28.073271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f05092ac-b5ff-4567-aaf9-080c1425d4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:28.081612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 997/1997] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:28.082321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05092ac-b5ff-4567-aaf9-080c1425d4c6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:28.083638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f05092ac-b5ff-4567-aaf9-080c1425d4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:28.095914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05092ac-b5ff-4567-aaf9-080c1425d4c6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.101526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc44adf-2b23-4312-830f-54d34250f600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.102361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1001/1998] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:28.102660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc44adf-2b23-4312-830f-54d34250f600 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.103224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc44adf-2b23-4312-830f-54d34250f600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.104261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc44adf-2b23-4312-830f-54d34250f600 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.104502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c306a1-e0e1-4925-ae49-6d4ba79f1d43 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.104989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c306a1-e0e1-4925-ae49-6d4ba79f1d43 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.105369 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc44adf-2b23-4312-830f-54d34250f600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.105477 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c306a1-e0e1-4925-ae49-6d4ba79f1d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.107216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc44adf-2b23-4312-830f-54d34250f600 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.126486 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 998/1999] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:28.127263 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc44adf-2b23-4312-830f-54d34250f600 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:28.144707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc44adf-2b23-4312-830f-54d34250f600 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.150386 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f4994-53b7-4866-99fb-b06969d1dd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.151254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f4994-53b7-4866-99fb-b06969d1dd7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.151660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f4994-53b7-4866-99fb-b06969d1dd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.152011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f4994-53b7-4866-99fb-b06969d1dd7e admin admin] PATH_INFO: `/v3/projects/528d667178324778bc25dd20bce66a12/users/88d3745e1dea492cb28b58809f2a421b/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.152564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f7c306a1-e0e1-4925-ae49-6d4ba79f1d43 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:28.154682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f4994-53b7-4866-99fb-b06969d1dd7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=528d667178324778bc25dd20bce66a12, user_id=88d3745e1dea492cb28b58809f2a421b, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.155317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f7c306a1-e0e1-4925-ae49-6d4ba79f1d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:28.157980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f4994-53b7-4866-99fb-b06969d1dd7e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.170415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1002/2000] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:28.171344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c306a1-e0e1-4925-ae49-6d4ba79f1d43 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:28.173886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 999/2001] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:28 2022] PUT /identity/v3/projects/528d667178324778bc25dd20bce66a12/users/88d3745e1dea492cb28b58809f2a421b/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:28.180630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f4994-53b7-4866-99fb-b06969d1dd7e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:28.189841 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c306a1-e0e1-4925-ae49-6d4ba79f1d43 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.195049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:28.206713 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:28.267175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f4994-53b7-4866-99fb-b06969d1dd7e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.272346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9345f6-8aac-4035-908e-edc6a6ef5a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.272984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9345f6-8aac-4035-908e-edc6a6ef5a46 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.273262 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9345f6-8aac-4035-908e-edc6a6ef5a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.273539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9345f6-8aac-4035-908e-edc6a6ef5a46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.274337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9345f6-8aac-4035-908e-edc6a6ef5a46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:24:28.274540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9345f6-8aac-4035-908e-edc6a6ef5a46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.276163 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9345f6-8aac-4035-908e-edc6a6ef5a46 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.286560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6c338-3c3f-4484-96b1-b88250309ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.288313 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6c338-3c3f-4484-96b1-b88250309ded admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.288814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6c338-3c3f-4484-96b1-b88250309ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.289342 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6c338-3c3f-4484-96b1-b88250309ded admin admin] PATH_INFO: `/v3/projects/97c0b8a086b046ddb2b2d475124dc692/users/415e8845ba8d46cc9dc939ef4556e865/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.294062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1000/2002] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:24:28.294062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a760d5-ddec-4319-b3b9-6e648007ece8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.294062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a760d5-ddec-4319-b3b9-6e648007ece8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.294062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a760d5-ddec-4319-b3b9-6e648007ece8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.324560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6c338-3c3f-4484-96b1-b88250309ded admin admin] RBAC: Authorizing `identity:create_grant(project_id=97c0b8a086b046ddb2b2d475124dc692, user_id=415e8845ba8d46cc9dc939ef4556e865, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.326122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-82a760d5-ddec-4319-b3b9-6e648007ece8 None None] MFA Rules not processed for user `88d3745e1dea492cb28b58809f2a421b`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:28.326122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-82a760d5-ddec-4319-b3b9-6e648007ece8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:28.330456 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6c338-3c3f-4484-96b1-b88250309ded admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.348946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1003/2003] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:28 2022] PUT /identity/v3/projects/97c0b8a086b046ddb2b2d475124dc692/users/415e8845ba8d46cc9dc939ef4556e865/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:28.349972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6c338-3c3f-4484-96b1-b88250309ded admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:28.437581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6c338-3c3f-4484-96b1-b88250309ded admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.441081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1001/2004] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:28 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:28.442511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1499c2e-7730-4dc2-a624-f9849b5f5568 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.442924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1499c2e-7730-4dc2-a624-f9849b5f5568 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.443059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc47a-523e-4379-98ad-a9719aa58335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.443392 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1499c2e-7730-4dc2-a624-f9849b5f5568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.443673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc47a-523e-4379-98ad-a9719aa58335 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.443986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc47a-523e-4379-98ad-a9719aa58335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.444264 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cc47a-523e-4379-98ad-a9719aa58335 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.444939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cc47a-523e-4379-98ad-a9719aa58335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.447020 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cc47a-523e-4379-98ad-a9719aa58335 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.466287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d1499c2e-7730-4dc2-a624-f9849b5f5568 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:28.468454 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d1499c2e-7730-4dc2-a624-f9849b5f5568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:28.493725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1004/2005] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:28 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:28.493725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc47a-523e-4379-98ad-a9719aa58335 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:28.494941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1002/2006] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:28.495116 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1499c2e-7730-4dc2-a624-f9849b5f5568 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:28.500652 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b03cc47a-523e-4379-98ad-a9719aa58335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:28.511319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1499c2e-7730-4dc2-a624-f9849b5f5568 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.589973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cc47a-523e-4379-98ad-a9719aa58335 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.595737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75445f7b-141d-4f09-ba92-8ef05c7677e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.596527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75445f7b-141d-4f09-ba92-8ef05c7677e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.596974 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75445f7b-141d-4f09-ba92-8ef05c7677e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.597408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75445f7b-141d-4f09-ba92-8ef05c7677e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.613501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d8cfa-aa78-40fd-8235-e8dd02fe1b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.614247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d8cfa-aa78-40fd-8235-e8dd02fe1b2e admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.614551 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d8cfa-aa78-40fd-8235-e8dd02fe1b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.614823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d8cfa-aa78-40fd-8235-e8dd02fe1b2e admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.615479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d8cfa-aa78-40fd-8235-e8dd02fe1b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.618250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d8cfa-aa78-40fd-8235-e8dd02fe1b2e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.638943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:28.642779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1003/2007] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:28 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:24:28.646879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d8cfa-aa78-40fd-8235-e8dd02fe1b2e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:28.648784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ac5d8cfa-aa78-40fd-8235-e8dd02fe1b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:28.681647 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:28.686144 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75445f7b-141d-4f09-ba92-8ef05c7677e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.687593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75445f7b-141d-4f09-ba92-8ef05c7677e4 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.699977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1005/2008] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:28.700882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75445f7b-141d-4f09-ba92-8ef05c7677e4 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:28.723633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75445f7b-141d-4f09-ba92-8ef05c7677e4 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.738840 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c5ec6-e7a3-4994-8cd7-c24fe64204fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.740322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c5ec6-e7a3-4994-8cd7-c24fe64204fe admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.740840 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c5ec6-e7a3-4994-8cd7-c24fe64204fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.741500 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c5ec6-e7a3-4994-8cd7-c24fe64204fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.742712 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c5ec6-e7a3-4994-8cd7-c24fe64204fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.742914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d8cfa-aa78-40fd-8235-e8dd02fe1b2e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.745699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c5ec6-e7a3-4994-8cd7-c24fe64204fe admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.748085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83588c4-5b6e-443e-8dc1-66a3f93892fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.748723 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83588c4-5b6e-443e-8dc1-66a3f93892fc service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.749001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83588c4-5b6e-443e-8dc1-66a3f93892fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.749276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83588c4-5b6e-443e-8dc1-66a3f93892fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.769363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1006/2009] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:28.770944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83588c4-5b6e-443e-8dc1-66a3f93892fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.773446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83588c4-5b6e-443e-8dc1-66a3f93892fc service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.776322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c5ec6-e7a3-4994-8cd7-c24fe64204fe admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:28.790761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c5ec6-e7a3-4994-8cd7-c24fe64204fe admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.796803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abd078-a4f2-4d52-b713-f300b1314134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.797438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abd078-a4f2-4d52-b713-f300b1314134 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.797714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abd078-a4f2-4d52-b713-f300b1314134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.798136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abd078-a4f2-4d52-b713-f300b1314134 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.798752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0abd078-a4f2-4d52-b713-f300b1314134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.801718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0abd078-a4f2-4d52-b713-f300b1314134 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.803388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1004/2010] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:28.815203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83588c4-5b6e-443e-8dc1-66a3f93892fc service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:28.831305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1007/2011] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:28.835672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abd078-a4f2-4d52-b713-f300b1314134 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:28.846905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83588c4-5b6e-443e-8dc1-66a3f93892fc service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.856908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2232d-f80c-4354-89af-121ac12b4105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.858330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2232d-f80c-4354-89af-121ac12b4105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.858450 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abd078-a4f2-4d52-b713-f300b1314134 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:28.858780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2232d-f80c-4354-89af-121ac12b4105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.859223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2232d-f80c-4354-89af-121ac12b4105 admin admin] PATH_INFO: `/v3/users/9efb2299c314422b85a733f09c7ff184` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.861767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2232d-f80c-4354-89af-121ac12b4105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9efb2299c314422b85a733f09c7ff184)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.866810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d2231a-401c-411a-88c4-276b84ca6b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:28.867023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2232d-f80c-4354-89af-121ac12b4105 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.868757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2231a-401c-411a-88c4-276b84ca6b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.869208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2231a-401c-411a-88c4-276b84ca6b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.869659 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d2231a-401c-411a-88c4-276b84ca6b8c admin admin] PATH_INFO: `/v3/projects/93a17d42b4214148a5417741adc02198/users/9aa93530363f4f28a2c437773dd25fea/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.905413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2231a-401c-411a-88c4-276b84ca6b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a17d42b4214148a5417741adc02198, user_id=9aa93530363f4f28a2c437773dd25fea, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:28.911375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d2231a-401c-411a-88c4-276b84ca6b8c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:28.958200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1008/2012] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:28 2022] PUT /identity/v3/projects/93a17d42b4214148a5417741adc02198/users/9aa93530363f4f28a2c437773dd25fea/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:28.964885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4452c-3f15-4893-a93a-06731ffb06b5 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:28.965358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4452c-3f15-4893-a93a-06731ffb06b5 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:28.965791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f4452c-3f15-4893-a93a-06731ffb06b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:28.968356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-21f2232d-f80c-4354-89af-121ac12b4105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9efb2299c314422b85a733f09c7ff184'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:28.979476 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-21f2232d-f80c-4354-89af-121ac12b4105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9efb2299c314422b85a733f09c7ff184'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:29.000893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-21f2232d-f80c-4354-89af-121ac12b4105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9efb2299c314422b85a733f09c7ff184'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:29.007414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-21f2232d-f80c-4354-89af-121ac12b4105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9efb2299c314422b85a733f09c7ff184'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:29.016873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a0f4452c-3f15-4893-a93a-06731ffb06b5 None None] MFA Rules not processed for user `9aa93530363f4f28a2c437773dd25fea`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:29.018580 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a0f4452c-3f15-4893-a93a-06731ffb06b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:29.023815 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-21f2232d-f80c-4354-89af-121ac12b4105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9efb2299c314422b85a733f09c7ff184'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:29.032341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1005/2013] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:28 2022] DELETE /identity/v3/users/9efb2299c314422b85a733f09c7ff184 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:29.039392 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b45b7-2ad0-4cdc-aefa-df598f6320a5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:29.039750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b45b7-2ad0-4cdc-aefa-df598f6320a5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:29.040187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b45b7-2ad0-4cdc-aefa-df598f6320a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:29.118133 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-754b45b7-2ad0-4cdc-aefa-df598f6320a5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:29.120194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1009/2014] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:29.120952 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-754b45b7-2ad0-4cdc-aefa-df598f6320a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:29.132886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4452c-3f15-4893-a93a-06731ffb06b5 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:29.134549 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a0f4452c-3f15-4893-a93a-06731ffb06b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:29.226720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1006/2015] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:29.238561 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b45b7-2ad0-4cdc-aefa-df598f6320a5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:29.240594 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-754b45b7-2ad0-4cdc-aefa-df598f6320a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:29.282245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f4452c-3f15-4893-a93a-06731ffb06b5 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:29.289358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:29.323813 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:29.372464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b45b7-2ad0-4cdc-aefa-df598f6320a5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:29.387665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920e621-a2b8-466b-bdf9-75ac1cdb8702 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:29.389204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920e621-a2b8-466b-bdf9-75ac1cdb8702 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:29.389853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920e621-a2b8-466b-bdf9-75ac1cdb8702 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:29.390529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a920e621-a2b8-466b-bdf9-75ac1cdb8702 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:29.426581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920e621-a2b8-466b-bdf9-75ac1cdb8702 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:29.436003 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a920e621-a2b8-466b-bdf9-75ac1cdb8702 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:29.450577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1007/2016] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:24:29.451074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79a133-36fc-4ceb-abcb-4bab75638083 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:29.451809 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79a133-36fc-4ceb-abcb-4bab75638083 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:29.452142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79a133-36fc-4ceb-abcb-4bab75638083 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:29.452538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79a133-36fc-4ceb-abcb-4bab75638083 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:29.557886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79a133-36fc-4ceb-abcb-4bab75638083 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:29.561522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79a133-36fc-4ceb-abcb-4bab75638083 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:29.580072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1010/2017] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:24:29.900439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920e621-a2b8-466b-bdf9-75ac1cdb8702 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:29.902830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a920e621-a2b8-466b-bdf9-75ac1cdb8702 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:29.917586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a920e621-a2b8-466b-bdf9-75ac1cdb8702 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:29.924163 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:29.925007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:29.925357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:29.926599 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] PATH_INFO: `/v3/users/9aa93530363f4f28a2c437773dd25fea` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:29.928515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa93530363f4f28a2c437773dd25fea)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:29.931485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:30.005500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa93530363f4f28a2c437773dd25fea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.020180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa93530363f4f28a2c437773dd25fea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.052656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6dac57-e70a-4bfa-af75-3d8ad1235977 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:30.052902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6dac57-e70a-4bfa-af75-3d8ad1235977 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:30.053223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6dac57-e70a-4bfa-af75-3d8ad1235977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:30.057744 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa93530363f4f28a2c437773dd25fea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.059022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa93530363f4f28a2c437773dd25fea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.090596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa93530363f4f28a2c437773dd25fea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.112816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1008/2018] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:29 2022] DELETE /identity/v3/users/9aa93530363f4f28a2c437773dd25fea => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:30.115629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:30.115629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:30.134285 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1f6dac57-e70a-4bfa-af75-3d8ad1235977 None None] MFA Rules not processed for user `415e8845ba8d46cc9dc939ef4556e865`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:30.136847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1f6dac57-e70a-4bfa-af75-3d8ad1235977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:30.229114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ff20e-7c5d-404b-ad5f-cc19b53984ac admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:30.235013 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:30.235895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:30.236245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:30.236589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] PATH_INFO: `/v3/projects/fc571556c1c045a99ec75bf7cbda9b91` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:30.238081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc571556c1c045a99ec75bf7cbda9b91)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:30.240412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:30.330363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1011/2019] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:30.331684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc571556c1c045a99ec75bf7cbda9b91', 'request_id': 'req-58db889d-f9db-4366-9e2c-d6dcd0639aa4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.332319 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af163c-f7c8-487e-b7de-46f8862f56f7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:30.334134 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:30.337578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af163c-f7c8-487e-b7de-46f8862f56f7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:30.338296 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af163c-f7c8-487e-b7de-46f8862f56f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:30.351483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc571556c1c045a99ec75bf7cbda9b91', 'request_id': 'req-58db889d-f9db-4366-9e2c-d6dcd0639aa4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.384513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc571556c1c045a99ec75bf7cbda9b91', 'request_id': 'req-58db889d-f9db-4366-9e2c-d6dcd0639aa4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.416770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] The token cache is being invalidate because project fc571556c1c045a99ec75bf7cbda9b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:30.417732 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.417932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-90af163c-f7c8-487e-b7de-46f8862f56f7 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:30.419912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.420161 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-90af163c-f7c8-487e-b7de-46f8862f56f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:30.427174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1009/2020] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:30 2022] DELETE /identity/v3/projects/fc571556c1c045a99ec75bf7cbda9b91 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:30.427174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:30.431280 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:30.498673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:30.546797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db889d-f9db-4366-9e2c-d6dcd0639aa4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:30.551697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1012/2021] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:30.552595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af163c-f7c8-487e-b7de-46f8862f56f7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:30.554174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa5f13-3603-4a24-b4a8-53bbe5d48aa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:30.555445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa5f13-3603-4a24-b4a8-53bbe5d48aa8 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:30.556146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa5f13-3603-4a24-b4a8-53bbe5d48aa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:30.556668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa5f13-3603-4a24-b4a8-53bbe5d48aa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:30.559509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-90af163c-f7c8-487e-b7de-46f8862f56f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:30.560034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d2fa5f13-3603-4a24-b4a8-53bbe5d48aa8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:30.573620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af163c-f7c8-487e-b7de-46f8862f56f7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:30.578959 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:30.579622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:30.580511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:30.580801 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] PATH_INFO: `/v3/users/88d3745e1dea492cb28b58809f2a421b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:30.582075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d3745e1dea492cb28b58809f2a421b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:30.584066 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:30.663972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d3745e1dea492cb28b58809f2a421b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.668793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa5f13-3603-4a24-b4a8-53bbe5d48aa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:30.672995 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa5f13-3603-4a24-b4a8-53bbe5d48aa8 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:30.675104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d3745e1dea492cb28b58809f2a421b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.676086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d3745e1dea492cb28b58809f2a421b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.688664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1010/2022] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:30.689831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa5f13-3603-4a24-b4a8-53bbe5d48aa8 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:30.691190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d3745e1dea492cb28b58809f2a421b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.692047 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d2fa5f13-3603-4a24-b4a8-53bbe5d48aa8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:30.717537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d3745e1dea492cb28b58809f2a421b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:30.726886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1013/2023] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:30 2022] DELETE /identity/v3/users/88d3745e1dea492cb28b58809f2a421b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:30.809621 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa5f13-3603-4a24-b4a8-53bbe5d48aa8 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:30.908939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:30.919771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9620f-af87-4f62-83a8-fcacb656d82b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:30.920818 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9620f-af87-4f62-83a8-fcacb656d82b service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:30.921835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9620f-af87-4f62-83a8-fcacb656d82b service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:30.922561 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9620f-af87-4f62-83a8-fcacb656d82b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:30.927177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d6b9620f-af87-4f62-83a8-fcacb656d82b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:31.076639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9620f-af87-4f62-83a8-fcacb656d82b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:31.078453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9620f-af87-4f62-83a8-fcacb656d82b service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:31.091632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1011/2024] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:31.291107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:31.293426 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:31.329876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f8007-9030-499c-8d74-747cc02cac2a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:31.338852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:31.339563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:31.339907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:31.340195 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5 admin admin] PATH_INFO: `/v3/projects/528d667178324778bc25dd20bce66a12` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:31.341531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=528d667178324778bc25dd20bce66a12)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:31.343930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:31.417531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '528d667178324778bc25dd20bce66a12', 'request_id': 'req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:31.431958 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '528d667178324778bc25dd20bce66a12', 'request_id': 'req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:31.447738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '528d667178324778bc25dd20bce66a12', 'request_id': 'req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:31.470457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5 admin admin] The token cache is being invalidate because project 528d667178324778bc25dd20bce66a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:31.470804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:31.471782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8d1a9763-8eae-4a7a-94b0-98af0dce10c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:31.474269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1014/2025] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:31 2022] DELETE /identity/v3/projects/528d667178324778bc25dd20bce66a12 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:31.483049 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65baa4d5-4a57-4d25-94b4-df238f97c51f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:31.483665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65baa4d5-4a57-4d25-94b4-df238f97c51f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:31.483665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65baa4d5-4a57-4d25-94b4-df238f97c51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:31.519233 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-65baa4d5-4a57-4d25-94b4-df238f97c51f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:31.525326 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-65baa4d5-4a57-4d25-94b4-df238f97c51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:31.643153 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1012/2026] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:31.653661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71449348-495f-4c86-82a6-670ac25e9963 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:31.654539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71449348-495f-4c86-82a6-670ac25e9963 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:31.655070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71449348-495f-4c86-82a6-670ac25e9963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:31.691015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-71449348-495f-4c86-82a6-670ac25e9963 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:31.693367 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-71449348-495f-4c86-82a6-670ac25e9963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:31.719007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1015/2027] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:31.725670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65baa4d5-4a57-4d25-94b4-df238f97c51f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:31.739948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71449348-495f-4c86-82a6-670ac25e9963 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:31.742199 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-71449348-495f-4c86-82a6-670ac25e9963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:31.747808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65baa4d5-4a57-4d25-94b4-df238f97c51f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:31.757133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71449348-495f-4c86-82a6-670ac25e9963 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:31.758434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81110e12-7ddd-444f-89eb-36f04e01bcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:31.760109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81110e12-7ddd-444f-89eb-36f04e01bcda admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:31.760695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81110e12-7ddd-444f-89eb-36f04e01bcda admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:31.761377 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81110e12-7ddd-444f-89eb-36f04e01bcda admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:31.762695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81110e12-7ddd-444f-89eb-36f04e01bcda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:24:31.763312 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81110e12-7ddd-444f-89eb-36f04e01bcda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:31.764105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:31.764952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:31.765324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:31.765725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] PATH_INFO: `/v3/projects/93a17d42b4214148a5417741adc02198` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:31.767127 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a17d42b4214148a5417741adc02198)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:31.767278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81110e12-7ddd-444f-89eb-36f04e01bcda admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:31.769323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:31.793593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:31.804288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1013/2028] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:24:31.809142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81110e12-7ddd-444f-89eb-36f04e01bcda admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:31.829712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81110e12-7ddd-444f-89eb-36f04e01bcda admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:31.837934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b179a-9e70-40e8-9800-09da85dddd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:31.838967 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b179a-9e70-40e8-9800-09da85dddd53 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:31.839445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b179a-9e70-40e8-9800-09da85dddd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:31.841056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b179a-9e70-40e8-9800-09da85dddd53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:31.842314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b179a-9e70-40e8-9800-09da85dddd53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:31.845827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b179a-9e70-40e8-9800-09da85dddd53 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:31.854603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a17d42b4214148a5417741adc02198', 'request_id': 'req-d59ae6d1-c064-43e6-a420-bd2bc607a012'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:31.863121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a17d42b4214148a5417741adc02198', 'request_id': 'req-d59ae6d1-c064-43e6-a420-bd2bc607a012'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:31.872951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a17d42b4214148a5417741adc02198', 'request_id': 'req-d59ae6d1-c064-43e6-a420-bd2bc607a012'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:31.895015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1014/2029] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:31 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:31.895942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] The token cache is being invalidate because project 93a17d42b4214148a5417741adc02198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:31.895942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:31.895942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:31.900532 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1016/2030] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:31 2022] DELETE /identity/v3/projects/93a17d42b4214148a5417741adc02198 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:31.902882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785b65b-21af-42dc-84a2-2c1e8da2d225 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:31.903435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785b65b-21af-42dc-84a2-2c1e8da2d225 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:31.903896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:31.904113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8785b65b-21af-42dc-84a2-2c1e8da2d225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:31.905824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:31.931156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8785b65b-21af-42dc-84a2-2c1e8da2d225 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:31.932817 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8785b65b-21af-42dc-84a2-2c1e8da2d225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:32.020595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1015/2031] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:32.023149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ae6d1-c064-43e6-a420-bd2bc607a012 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:32.028461 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44230730-9665-471c-95eb-6fa520d6a9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:32.029114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44230730-9665-471c-95eb-6fa520d6a9ce admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.029396 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44230730-9665-471c-95eb-6fa520d6a9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.029696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44230730-9665-471c-95eb-6fa520d6a9ce admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.030490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44230730-9665-471c-95eb-6fa520d6a9ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:32.030636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17758c95-7bff-47ff-b186-eeed920de83a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.030988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17758c95-7bff-47ff-b186-eeed920de83a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.031176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17758c95-7bff-47ff-b186-eeed920de83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.032754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44230730-9665-471c-95eb-6fa520d6a9ce admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:32.041585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:32.057068 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-17758c95-7bff-47ff-b186-eeed920de83a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:32.058985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-17758c95-7bff-47ff-b186-eeed920de83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:32.060603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1017/2032] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:31 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:32.065729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44230730-9665-471c-95eb-6fa520d6a9ce admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:32.069182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1016/2033] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:32.073841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17758c95-7bff-47ff-b186-eeed920de83a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:32.083720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17758c95-7bff-47ff-b186-eeed920de83a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:32.086655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44230730-9665-471c-95eb-6fa520d6a9ce admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:32.089194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32897314-c9b8-4bbe-8212-ae281e0e2feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:32.089820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32897314-c9b8-4bbe-8212-ae281e0e2feb admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.090140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32897314-c9b8-4bbe-8212-ae281e0e2feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.090414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32897314-c9b8-4bbe-8212-ae281e0e2feb admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.091076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32897314-c9b8-4bbe-8212-ae281e0e2feb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:24:32.091379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32897314-c9b8-4bbe-8212-ae281e0e2feb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:32.092998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32897314-c9b8-4bbe-8212-ae281e0e2feb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:32.096146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e259a-4e81-4240-8fa5-0f2ea8b64983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:32.099101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e259a-4e81-4240-8fa5-0f2ea8b64983 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.099919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e259a-4e81-4240-8fa5-0f2ea8b64983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.100820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e259a-4e81-4240-8fa5-0f2ea8b64983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.103122 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e259a-4e81-4240-8fa5-0f2ea8b64983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:32.107520 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e259a-4e81-4240-8fa5-0f2ea8b64983 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:32.113834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1017/2034] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:24:32.115031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32897314-c9b8-4bbe-8212-ae281e0e2feb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:32.127454 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32897314-c9b8-4bbe-8212-ae281e0e2feb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:32.130352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1018/2035] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:24:32.133479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e733d-171e-4f99-a94c-bec10290b190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:32.134167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e733d-171e-4f99-a94c-bec10290b190 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.134722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e733d-171e-4f99-a94c-bec10290b190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.134722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e733d-171e-4f99-a94c-bec10290b190 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.135331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e733d-171e-4f99-a94c-bec10290b190 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:32.136308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e259a-4e81-4240-8fa5-0f2ea8b64983 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:32.137282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e733d-171e-4f99-a94c-bec10290b190 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:32.154012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e259a-4e81-4240-8fa5-0f2ea8b64983 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:32.162830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43fef1-f3d8-4647-8542-2502a3939251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:32.163490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43fef1-f3d8-4647-8542-2502a3939251 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.163819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43fef1-f3d8-4647-8542-2502a3939251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.164252 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43fef1-f3d8-4647-8542-2502a3939251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.165189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43fef1-f3d8-4647-8542-2502a3939251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:32.167949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43fef1-f3d8-4647-8542-2502a3939251 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:32.171112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1018/2036] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:32 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:32.175824 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e733d-171e-4f99-a94c-bec10290b190 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:32.192495 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1019/2037] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:32.194792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43fef1-f3d8-4647-8542-2502a3939251 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:32.210704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43fef1-f3d8-4647-8542-2502a3939251 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:32.261455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e733d-171e-4f99-a94c-bec10290b190 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:32.269368 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e9e39-3667-4fca-b0e7-6c7be5b62f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:32.270290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e9e39-3667-4fca-b0e7-6c7be5b62f3c admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.270689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e9e39-3667-4fca-b0e7-6c7be5b62f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.271100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e9e39-3667-4fca-b0e7-6c7be5b62f3c admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.272828 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e9e39-3667-4fca-b0e7-6c7be5b62f3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:32.277077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e9e39-3667-4fca-b0e7-6c7be5b62f3c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:32.299562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ae508-d69b-44f5-b946-1b4dd85e4ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:32.300603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ae508-d69b-44f5-b946-1b4dd85e4ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.301061 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ae508-d69b-44f5-b946-1b4dd85e4ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.301507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ae508-d69b-44f5-b946-1b4dd85e4ef8 admin admin] PATH_INFO: `/v3/projects/0c4953f71d154923b3181877dbf7e195/users/5b8d7610c56147cca292e2552fd25cc9/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.310924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1019/2038] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:32 2022] POST /identity/v3/users => generated 296 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:24:32.316660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e9e39-3667-4fca-b0e7-6c7be5b62f3c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:32.329217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ae508-d69b-44f5-b946-1b4dd85e4ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4953f71d154923b3181877dbf7e195, user_id=5b8d7610c56147cca292e2552fd25cc9, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:32.331450 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e9e39-3667-4fca-b0e7-6c7be5b62f3c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:32.335004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ae508-d69b-44f5-b946-1b4dd85e4ef8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:32.348167 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:32.351425 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8ef7d-62f7-4c69-805e-db8234215bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:32.351646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8ef7d-62f7-4c69-805e-db8234215bac admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.352313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8ef7d-62f7-4c69-805e-db8234215bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.353521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8ef7d-62f7-4c69-805e-db8234215bac admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.360265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8ef7d-62f7-4c69-805e-db8234215bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:32.363895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8ef7d-62f7-4c69-805e-db8234215bac admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:32.374174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1020/2039] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:32 2022] PUT /identity/v3/projects/0c4953f71d154923b3181877dbf7e195/users/5b8d7610c56147cca292e2552fd25cc9/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:32.377204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac472883-900e-48d4-8544-80579021d7e8 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.377732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac472883-900e-48d4-8544-80579021d7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.378245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac472883-900e-48d4-8544-80579021d7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.386094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1020/2040] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:24:32.387101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa30839-3151-4b57-88e8-deccc0670bec None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.387975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa30839-3151-4b57-88e8-deccc0670bec None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.388726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa30839-3151-4b57-88e8-deccc0670bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.407593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ac472883-900e-48d4-8544-80579021d7e8 None None] MFA Rules not processed for user `415e8845ba8d46cc9dc939ef4556e865`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:32.411058 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ac472883-900e-48d4-8544-80579021d7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:32.435993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5aa30839-3151-4b57-88e8-deccc0670bec None None] MFA Rules not processed for user `5b8d7610c56147cca292e2552fd25cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:32.438301 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5aa30839-3151-4b57-88e8-deccc0670bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:32.494908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1021/2041] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:32.496098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac472883-900e-48d4-8544-80579021d7e8 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:32.563623 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1021/2042] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:32.578303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa30839-3151-4b57-88e8-deccc0670bec None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:32.580022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5aa30839-3151-4b57-88e8-deccc0670bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:32.631181 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac472883-900e-48d4-8544-80579021d7e8 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:32.643342 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327a9fa-eb5f-4a49-b9c9-c85dc5fc767b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:32.644279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a9fa-eb5f-4a49-b9c9-c85dc5fc767b admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.644808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a9fa-eb5f-4a49-b9c9-c85dc5fc767b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.645675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327a9fa-eb5f-4a49-b9c9-c85dc5fc767b admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.646172 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327a9fa-eb5f-4a49-b9c9-c85dc5fc767b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:32.648235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327a9fa-eb5f-4a49-b9c9-c85dc5fc767b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:32.650386 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:32.665248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1022/2043] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:32.669662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327a9fa-eb5f-4a49-b9c9-c85dc5fc767b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:32.681806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327a9fa-eb5f-4a49-b9c9-c85dc5fc767b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:32.686972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51025322-8a5b-4ba0-9efd-a367d49d2ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:32.687745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51025322-8a5b-4ba0-9efd-a367d49d2ecb admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.688057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51025322-8a5b-4ba0-9efd-a367d49d2ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.688369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51025322-8a5b-4ba0-9efd-a367d49d2ecb admin admin] PATH_INFO: `/v3/projects/d92349cf06e34dcdaa7df2e64aa1cb31/users/60a37e2c2f9f4024bc5b52e6b02332d9/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.706142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa30839-3151-4b57-88e8-deccc0670bec None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:32.707911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51025322-8a5b-4ba0-9efd-a367d49d2ecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92349cf06e34dcdaa7df2e64aa1cb31, user_id=60a37e2c2f9f4024bc5b52e6b02332d9, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:32.711429 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51025322-8a5b-4ba0-9efd-a367d49d2ecb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:32.713884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15ef03-6f7e-4ca3-aa50-f4f6581a2199 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:32.715031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15ef03-6f7e-4ca3-aa50-f4f6581a2199 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.715511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15ef03-6f7e-4ca3-aa50-f4f6581a2199 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.716197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15ef03-6f7e-4ca3-aa50-f4f6581a2199 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.733769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1023/2044] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:32 2022] PUT /identity/v3/projects/d92349cf06e34dcdaa7df2e64aa1cb31/users/60a37e2c2f9f4024bc5b52e6b02332d9/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:32.735566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15ef03-6f7e-4ca3-aa50-f4f6581a2199 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:32.738545 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15ef03-6f7e-4ca3-aa50-f4f6581a2199 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:32.746102 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a276f-497f-4171-bb90-a2019dc07660 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:32.747141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a276f-497f-4171-bb90-a2019dc07660 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:32.749542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a276f-497f-4171-bb90-a2019dc07660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:32.776066 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1022/2045] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:32.844608 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-584a276f-497f-4171-bb90-a2019dc07660 None None] MFA Rules not processed for user `60a37e2c2f9f4024bc5b52e6b02332d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:32.850077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-584a276f-497f-4171-bb90-a2019dc07660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:32.961631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1024/2046] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:32.966745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15ef03-6f7e-4ca3-aa50-f4f6581a2199 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:32.990650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15ef03-6f7e-4ca3-aa50-f4f6581a2199 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:33.091487 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83306f18-bd3d-46fe-beb0-d8d76f4a28a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:33.092258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83306f18-bd3d-46fe-beb0-d8d76f4a28a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:33.092547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83306f18-bd3d-46fe-beb0-d8d76f4a28a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:33.092823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83306f18-bd3d-46fe-beb0-d8d76f4a28a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:33.093484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83306f18-bd3d-46fe-beb0-d8d76f4a28a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:33.095851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83306f18-bd3d-46fe-beb0-d8d76f4a28a1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:33.162943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1023/2047] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:32 2022] POST /identity/v3/projects => generated 393 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:33.169241 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a276f-497f-4171-bb90-a2019dc07660 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:33.175846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:33.184737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a276f-497f-4171-bb90-a2019dc07660 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:33.299397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae40c10-28be-43b2-8654-a5bbee730f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:33.301162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae40c10-28be-43b2-8654-a5bbee730f27 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:33.302219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae40c10-28be-43b2-8654-a5bbee730f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:33.302920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae40c10-28be-43b2-8654-a5bbee730f27 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:33.304558 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae40c10-28be-43b2-8654-a5bbee730f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:33.308983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae40c10-28be-43b2-8654-a5bbee730f27 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:33.362108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1025/2048] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:33 2022] POST /identity/v3/users => generated 303 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:33.368531 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83306f18-bd3d-46fe-beb0-d8d76f4a28a1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:33.414994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83306f18-bd3d-46fe-beb0-d8d76f4a28a1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:33.421559 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f652b96-70aa-4e56-8739-b04a8163a817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:33.423231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f652b96-70aa-4e56-8739-b04a8163a817 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:33.423882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f652b96-70aa-4e56-8739-b04a8163a817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:33.424206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f652b96-70aa-4e56-8739-b04a8163a817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:33.425377 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f652b96-70aa-4e56-8739-b04a8163a817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:33.427698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f652b96-70aa-4e56-8739-b04a8163a817 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:33.456920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1024/2049] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:33.462511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae40c10-28be-43b2-8654-a5bbee730f27 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:33.475449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae40c10-28be-43b2-8654-a5bbee730f27 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:33.480523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27318e-be23-4a1d-a487-e253433b46a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:33.481213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27318e-be23-4a1d-a487-e253433b46a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:33.481502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27318e-be23-4a1d-a487-e253433b46a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:33.481802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27318e-be23-4a1d-a487-e253433b46a9 admin admin] PATH_INFO: `/v3/projects/99f7a6b8fb9f4dccb036d268c607f2ef/users/e595baf8b00b454cafda68ab37cb8cf3/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:33.510813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27318e-be23-4a1d-a487-e253433b46a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99f7a6b8fb9f4dccb036d268c607f2ef, user_id=e595baf8b00b454cafda68ab37cb8cf3, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:33.519282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27318e-be23-4a1d-a487-e253433b46a9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:33.545117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1026/2050] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:33 2022] PUT /identity/v3/projects/99f7a6b8fb9f4dccb036d268c607f2ef/users/e595baf8b00b454cafda68ab37cb8cf3/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:33.549682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f652b96-70aa-4e56-8739-b04a8163a817 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:33.569568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f652b96-70aa-4e56-8739-b04a8163a817 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:33.682851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628513f-2585-45d1-92be-e85767347684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:33.683654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628513f-2585-45d1-92be-e85767347684 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:33.683771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628513f-2585-45d1-92be-e85767347684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:33.684032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628513f-2585-45d1-92be-e85767347684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:33.684708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628513f-2585-45d1-92be-e85767347684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:33.686846 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628513f-2585-45d1-92be-e85767347684 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:33.702331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1025/2051] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:33 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:33.705975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27318e-be23-4a1d-a487-e253433b46a9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:33.719536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27318e-be23-4a1d-a487-e253433b46a9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:33.731085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2781a-fd6e-4882-88bb-fdbaa0a5dfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:33.732238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2781a-fd6e-4882-88bb-fdbaa0a5dfa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:33.732719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2781a-fd6e-4882-88bb-fdbaa0a5dfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:33.733192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c2781a-fd6e-4882-88bb-fdbaa0a5dfa6 admin admin] PATH_INFO: `/v3/projects/99f7a6b8fb9f4dccb036d268c607f2ef/users/e595baf8b00b454cafda68ab37cb8cf3/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:33.737908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2781a-fd6e-4882-88bb-fdbaa0a5dfa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99f7a6b8fb9f4dccb036d268c607f2ef, user_id=e595baf8b00b454cafda68ab37cb8cf3, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:33.743216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c2781a-fd6e-4882-88bb-fdbaa0a5dfa6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:33.757599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1027/2052] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:33 2022] PUT /identity/v3/projects/99f7a6b8fb9f4dccb036d268c607f2ef/users/e595baf8b00b454cafda68ab37cb8cf3/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:33.768795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc61f2-c9ee-4197-b1c8-7d5a7b6a2340 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:33.769211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc61f2-c9ee-4197-b1c8-7d5a7b6a2340 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:33.769659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbc61f2-c9ee-4197-b1c8-7d5a7b6a2340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:33.812101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-afbc61f2-c9ee-4197-b1c8-7d5a7b6a2340 None None] MFA Rules not processed for user `e595baf8b00b454cafda68ab37cb8cf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:33.814434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-afbc61f2-c9ee-4197-b1c8-7d5a7b6a2340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:33.857896 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:33.947874 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1026/2053] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:33.960385 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2781a-fd6e-4882-88bb-fdbaa0a5dfa6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:33.962912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-90c2781a-fd6e-4882-88bb-fdbaa0a5dfa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:34.043393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c2781a-fd6e-4882-88bb-fdbaa0a5dfa6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:34.047602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaceb1c-e3d6-406d-acfb-1b1b7cde578e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:34.048231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaceb1c-e3d6-406d-acfb-1b1b7cde578e service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:34.048505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaceb1c-e3d6-406d-acfb-1b1b7cde578e service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:34.048792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaceb1c-e3d6-406d-acfb-1b1b7cde578e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:34.061888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaceb1c-e3d6-406d-acfb-1b1b7cde578e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:34.063435 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaceb1c-e3d6-406d-acfb-1b1b7cde578e service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:34.075803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1028/2054] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:33 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:34.176968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc61f2-c9ee-4197-b1c8-7d5a7b6a2340 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:34.192551 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbc61f2-c9ee-4197-b1c8-7d5a7b6a2340 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:34.197115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5811a-4682-4297-8042-78f6568ea9b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:34.197754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5811a-4682-4297-8042-78f6568ea9b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:34.198037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5811a-4682-4297-8042-78f6568ea9b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:34.198354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5811a-4682-4297-8042-78f6568ea9b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:34.325496 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5811a-4682-4297-8042-78f6568ea9b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:34.328509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5811a-4682-4297-8042-78f6568ea9b7 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:34.348030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1027/2055] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:39.750175 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaceb1c-e3d6-406d-acfb-1b1b7cde578e service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:39.839466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:39.845139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaceb1c-e3d6-406d-acfb-1b1b7cde578e service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:39.855795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:39.857069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:39.857433 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:39.857949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] PATH_INFO: `/v3/users/60a37e2c2f9f4024bc5b52e6b02332d9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:39.860196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60a37e2c2f9f4024bc5b52e6b02332d9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:39.864642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:39.961594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60a37e2c2f9f4024bc5b52e6b02332d9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:39.975305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60a37e2c2f9f4024bc5b52e6b02332d9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:39.975901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60a37e2c2f9f4024bc5b52e6b02332d9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:39.986761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60a37e2c2f9f4024bc5b52e6b02332d9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:40.006484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60a37e2c2f9f4024bc5b52e6b02332d9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:40.016360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1029/2056] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:39 2022] DELETE /identity/v3/users/60a37e2c2f9f4024bc5b52e6b02332d9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:40.022746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35307a3-0b6a-43cf-81cc-d4eea92fc285 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:40.023620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35307a3-0b6a-43cf-81cc-d4eea92fc285 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:40.023620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35307a3-0b6a-43cf-81cc-d4eea92fc285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:40.052604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e35307a3-0b6a-43cf-81cc-d4eea92fc285 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:40.054548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e35307a3-0b6a-43cf-81cc-d4eea92fc285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:40.149152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1028/2057] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:40.164091 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:40.166490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:40.270603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd750d5-d018-49bc-a270-a2e24b665f51 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:40.275193 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edf18e-49ec-4f95-93fe-097fb1521236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:40.276040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edf18e-49ec-4f95-93fe-097fb1521236 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:40.276409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edf18e-49ec-4f95-93fe-097fb1521236 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:40.276767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edf18e-49ec-4f95-93fe-097fb1521236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:40.288514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edf18e-49ec-4f95-93fe-097fb1521236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:40.290254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edf18e-49ec-4f95-93fe-097fb1521236 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:40.303552 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1030/2058] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:40.766973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35307a3-0b6a-43cf-81cc-d4eea92fc285 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:40.770089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e35307a3-0b6a-43cf-81cc-d4eea92fc285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:40.793427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35307a3-0b6a-43cf-81cc-d4eea92fc285 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:40.803433 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:40.804666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:40.805276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:40.805857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] PATH_INFO: `/v3/projects/d92349cf06e34dcdaa7df2e64aa1cb31` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:40.808250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92349cf06e34dcdaa7df2e64aa1cb31)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:40.812324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:40.900026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92349cf06e34dcdaa7df2e64aa1cb31', 'request_id': 'req-eb782d84-421d-4b07-9cf2-e6ff28267295'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:40.913127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92349cf06e34dcdaa7df2e64aa1cb31', 'request_id': 'req-eb782d84-421d-4b07-9cf2-e6ff28267295'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:40.930441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:40.933234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92349cf06e34dcdaa7df2e64aa1cb31', 'request_id': 'req-eb782d84-421d-4b07-9cf2-e6ff28267295'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:40.961627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] The token cache is being invalidate because project d92349cf06e34dcdaa7df2e64aa1cb31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:40.962653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:40.964275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:40.968074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1029/2059] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:40 2022] DELETE /identity/v3/projects/d92349cf06e34dcdaa7df2e64aa1cb31 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:40.975518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edf18e-49ec-4f95-93fe-097fb1521236 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:40.990525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edf18e-49ec-4f95-93fe-097fb1521236 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:41.060179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e18a2a9-1f36-4117-b057-867a44f8cdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:41.060909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a2a9-1f36-4117-b057-867a44f8cdfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:41.061256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a2a9-1f36-4117-b057-867a44f8cdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:41.061594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e18a2a9-1f36-4117-b057-867a44f8cdfd admin admin] PATH_INFO: `/v3/users/e595baf8b00b454cafda68ab37cb8cf3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:41.063010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18a2a9-1f36-4117-b057-867a44f8cdfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e595baf8b00b454cafda68ab37cb8cf3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:41.065155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e18a2a9-1f36-4117-b057-867a44f8cdfd admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:41.093124 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:41.142474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0e18a2a9-1f36-4117-b057-867a44f8cdfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e595baf8b00b454cafda68ab37cb8cf3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:41.153245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0e18a2a9-1f36-4117-b057-867a44f8cdfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e595baf8b00b454cafda68ab37cb8cf3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:41.154136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0e18a2a9-1f36-4117-b057-867a44f8cdfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e595baf8b00b454cafda68ab37cb8cf3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:41.168118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0e18a2a9-1f36-4117-b057-867a44f8cdfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e595baf8b00b454cafda68ab37cb8cf3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:41.192087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0e18a2a9-1f36-4117-b057-867a44f8cdfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e595baf8b00b454cafda68ab37cb8cf3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:41.200299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1031/2060] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:40 2022] DELETE /identity/v3/users/e595baf8b00b454cafda68ab37cb8cf3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:41.547730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:41.549666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:41.566814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb782d84-421d-4b07-9cf2-e6ff28267295 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:41.640068 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82612c07-15de-411e-b4ea-bbf735d7ba0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:41.640749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82612c07-15de-411e-b4ea-bbf735d7ba0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:41.641136 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82612c07-15de-411e-b4ea-bbf735d7ba0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:41.641451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82612c07-15de-411e-b4ea-bbf735d7ba0a admin admin] PATH_INFO: `/v3/projects/99f7a6b8fb9f4dccb036d268c607f2ef` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:41.642837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82612c07-15de-411e-b4ea-bbf735d7ba0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=99f7a6b8fb9f4dccb036d268c607f2ef)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:41.645116 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82612c07-15de-411e-b4ea-bbf735d7ba0a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:41.718432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82612c07-15de-411e-b4ea-bbf735d7ba0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99f7a6b8fb9f4dccb036d268c607f2ef', 'request_id': 'req-82612c07-15de-411e-b4ea-bbf735d7ba0a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:41.730702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82612c07-15de-411e-b4ea-bbf735d7ba0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99f7a6b8fb9f4dccb036d268c607f2ef', 'request_id': 'req-82612c07-15de-411e-b4ea-bbf735d7ba0a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:41.737259 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82612c07-15de-411e-b4ea-bbf735d7ba0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99f7a6b8fb9f4dccb036d268c607f2ef', 'request_id': 'req-82612c07-15de-411e-b4ea-bbf735d7ba0a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:41.751442 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82612c07-15de-411e-b4ea-bbf735d7ba0a admin admin] The token cache is being invalidate because project 99f7a6b8fb9f4dccb036d268c607f2ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:41.751788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82612c07-15de-411e-b4ea-bbf735d7ba0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:41.752577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82612c07-15de-411e-b4ea-bbf735d7ba0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:41.754271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1030/2061] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:41 2022] DELETE /identity/v3/projects/99f7a6b8fb9f4dccb036d268c607f2ef => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:41.759509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0c9c3-0425-448f-8a4a-5105c6c2af7c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:41.759861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0c9c3-0425-448f-8a4a-5105c6c2af7c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:41.760140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e0c9c3-0425-448f-8a4a-5105c6c2af7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:41.782267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f7e0c9c3-0425-448f-8a4a-5105c6c2af7c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:41.783648 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f7e0c9c3-0425-448f-8a4a-5105c6c2af7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:41.854375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1032/2062] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:41.863118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912ad2a-3543-4fac-a9a9-d5626190073b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:41.863624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912ad2a-3543-4fac-a9a9-d5626190073b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:41.864035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912ad2a-3543-4fac-a9a9-d5626190073b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:41.898420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f912ad2a-3543-4fac-a9a9-d5626190073b None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:41.900540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f912ad2a-3543-4fac-a9a9-d5626190073b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:41.914608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1031/2063] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:41.919893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0c9c3-0425-448f-8a4a-5105c6c2af7c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:41.930552 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e0c9c3-0425-448f-8a4a-5105c6c2af7c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:41.936046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ab0de-5618-46f2-8b80-f680d2eaf9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:41.936673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ab0de-5618-46f2-8b80-f680d2eaf9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:41.936952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ab0de-5618-46f2-8b80-f680d2eaf9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:41.937226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ab0de-5618-46f2-8b80-f680d2eaf9b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:41.937904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ab0de-5618-46f2-8b80-f680d2eaf9b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:24:41.938242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ab0de-5618-46f2-8b80-f680d2eaf9b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:41.939989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ab0de-5618-46f2-8b80-f680d2eaf9b1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:41.954283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1033/2064] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:24:41.960773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912ad2a-3543-4fac-a9a9-d5626190073b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:41.972501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912ad2a-3543-4fac-a9a9-d5626190073b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:41.979449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bc4f7-5b83-4a41-9472-bae19e9f8efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:41.980081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bc4f7-5b83-4a41-9472-bae19e9f8efc admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:41.980380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bc4f7-5b83-4a41-9472-bae19e9f8efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:41.980657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6bc4f7-5b83-4a41-9472-bae19e9f8efc admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:41.981327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bc4f7-5b83-4a41-9472-bae19e9f8efc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:41.983491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6bc4f7-5b83-4a41-9472-bae19e9f8efc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:42.018294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1032/2065] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:41 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:42.024124 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ab0de-5618-46f2-8b80-f680d2eaf9b1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:42.042545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ab0de-5618-46f2-8b80-f680d2eaf9b1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:42.134120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8789344-553c-4bc3-8690-5b3661dc5c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:42.135075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8789344-553c-4bc3-8690-5b3661dc5c63 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:42.135582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8789344-553c-4bc3-8690-5b3661dc5c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:42.136072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8789344-553c-4bc3-8690-5b3661dc5c63 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:42.137215 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8789344-553c-4bc3-8690-5b3661dc5c63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:42.141073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8789344-553c-4bc3-8690-5b3661dc5c63 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:42.178466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1034/2066] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:42 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:42.180103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bc4f7-5b83-4a41-9472-bae19e9f8efc admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:42.196356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6bc4f7-5b83-4a41-9472-bae19e9f8efc admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:42.205931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddb542-77ca-45c4-ae09-44d8e02dcbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:42.207326 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddb542-77ca-45c4-ae09-44d8e02dcbad admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:42.207993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddb542-77ca-45c4-ae09-44d8e02dcbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:42.209089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ddb542-77ca-45c4-ae09-44d8e02dcbad admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:42.210321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ddb542-77ca-45c4-ae09-44d8e02dcbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:42.216300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ddb542-77ca-45c4-ae09-44d8e02dcbad admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:42.234104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1033/2067] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:24:42.236752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8789344-553c-4bc3-8690-5b3661dc5c63 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:42.249064 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8789344-553c-4bc3-8690-5b3661dc5c63 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:42.254394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024649b8-4b3f-4d05-9422-f8b6a7064fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:42.255020 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024649b8-4b3f-4d05-9422-f8b6a7064fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:42.255300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024649b8-4b3f-4d05-9422-f8b6a7064fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:42.256541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024649b8-4b3f-4d05-9422-f8b6a7064fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:42.257140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024649b8-4b3f-4d05-9422-f8b6a7064fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:42.259334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024649b8-4b3f-4d05-9422-f8b6a7064fe7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:42.272667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1035/2068] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:24:42.277464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddb542-77ca-45c4-ae09-44d8e02dcbad admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:42.293128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ddb542-77ca-45c4-ae09-44d8e02dcbad admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:42.298666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15565ce-848f-4a7d-b8fc-77b03872a9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:42.299357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15565ce-848f-4a7d-b8fc-77b03872a9a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:42.299636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15565ce-848f-4a7d-b8fc-77b03872a9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:42.299912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15565ce-848f-4a7d-b8fc-77b03872a9a7 admin admin] PATH_INFO: `/v3/projects/8c366b91687f434e944d21904600f567/users/625e0595518d45d3b564f6d9a1d48cb0/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:42.332477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15565ce-848f-4a7d-b8fc-77b03872a9a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c366b91687f434e944d21904600f567, user_id=625e0595518d45d3b564f6d9a1d48cb0, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:42.339736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15565ce-848f-4a7d-b8fc-77b03872a9a7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:42.360690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1034/2069] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:42 2022] PUT /identity/v3/projects/8c366b91687f434e944d21904600f567/users/625e0595518d45d3b564f6d9a1d48cb0/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:42.366272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58642e9-53f2-4a23-ab8c-c9049639e2df None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:42.366565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58642e9-53f2-4a23-ab8c-c9049639e2df None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:42.366844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58642e9-53f2-4a23-ab8c-c9049639e2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:42.381059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:42.405968 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c58642e9-53f2-4a23-ab8c-c9049639e2df None None] MFA Rules not processed for user `625e0595518d45d3b564f6d9a1d48cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:42.407542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c58642e9-53f2-4a23-ab8c-c9049639e2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:42.493917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1036/2070] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:42.500388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15565ce-848f-4a7d-b8fc-77b03872a9a7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:42.528670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:42.590471 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15565ce-848f-4a7d-b8fc-77b03872a9a7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:42.596408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f7bb1-1b93-4971-a92b-056d3275e160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:42.597323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f7bb1-1b93-4971-a92b-056d3275e160 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:42.597775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f7bb1-1b93-4971-a92b-056d3275e160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:42.598256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f7bb1-1b93-4971-a92b-056d3275e160 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:42.599005 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f7bb1-1b93-4971-a92b-056d3275e160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:42.601153 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f7bb1-1b93-4971-a92b-056d3275e160 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:42.635607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1035/2071] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:42 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:24:42.640307 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58642e9-53f2-4a23-ab8c-c9049639e2df None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:42.654271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58642e9-53f2-4a23-ab8c-c9049639e2df None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:42.757127 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74bd92-810b-4eff-b95e-ae712d8f12d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:42.758204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74bd92-810b-4eff-b95e-ae712d8f12d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:42.758579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74bd92-810b-4eff-b95e-ae712d8f12d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:42.758938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d74bd92-810b-4eff-b95e-ae712d8f12d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:42.759755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74bd92-810b-4eff-b95e-ae712d8f12d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:42.763177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d74bd92-810b-4eff-b95e-ae712d8f12d1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:42.791305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1037/2072] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:42 2022] POST /identity/v3/users => generated 298 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:42.795672 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f7bb1-1b93-4971-a92b-056d3275e160 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:42.807718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f7bb1-1b93-4971-a92b-056d3275e160 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:42.812662 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3145f2-2eb6-4690-b690-24252dab9246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:42.813291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3145f2-2eb6-4690-b690-24252dab9246 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:42.813576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3145f2-2eb6-4690-b690-24252dab9246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:42.813846 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3145f2-2eb6-4690-b690-24252dab9246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:42.814542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3145f2-2eb6-4690-b690-24252dab9246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:42.816154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3145f2-2eb6-4690-b690-24252dab9246 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:42.828733 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1036/2073] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:42.835569 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74bd92-810b-4eff-b95e-ae712d8f12d1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:42.847050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d74bd92-810b-4eff-b95e-ae712d8f12d1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:42.852054 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44707b2c-26de-4ba9-80c7-7c2c4493074f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:42.852734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44707b2c-26de-4ba9-80c7-7c2c4493074f admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:42.853014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44707b2c-26de-4ba9-80c7-7c2c4493074f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:42.853288 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44707b2c-26de-4ba9-80c7-7c2c4493074f admin admin] PATH_INFO: `/v3/projects/02d677c3bdea49bd8454f301e29a2615/users/43ac920dd1cf41a1a18df8b738cc88a6/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:42.870483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44707b2c-26de-4ba9-80c7-7c2c4493074f admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d677c3bdea49bd8454f301e29a2615, user_id=43ac920dd1cf41a1a18df8b738cc88a6, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:42.873195 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44707b2c-26de-4ba9-80c7-7c2c4493074f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:42.889435 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1038/2074] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:42 2022] PUT /identity/v3/projects/02d677c3bdea49bd8454f301e29a2615/users/43ac920dd1cf41a1a18df8b738cc88a6/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:42.893489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3145f2-2eb6-4690-b690-24252dab9246 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:42.904144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3145f2-2eb6-4690-b690-24252dab9246 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:42.977090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f617cc37-d2d8-41c7-959c-dd0d9303d8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:42.977624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617cc37-d2d8-41c7-959c-dd0d9303d8ab admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:42.977900 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617cc37-d2d8-41c7-959c-dd0d9303d8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:42.978207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617cc37-d2d8-41c7-959c-dd0d9303d8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:42.978863 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617cc37-d2d8-41c7-959c-dd0d9303d8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:42.980477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f617cc37-d2d8-41c7-959c-dd0d9303d8ab admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:42.993340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1037/2075] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:42.998565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44707b2c-26de-4ba9-80c7-7c2c4493074f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:43.010423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44707b2c-26de-4ba9-80c7-7c2c4493074f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:43.015298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa989a17-0d6b-4822-9b3b-d2ded81dc1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:43.015991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa989a17-0d6b-4822-9b3b-d2ded81dc1c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:43.016347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa989a17-0d6b-4822-9b3b-d2ded81dc1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:43.016542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa989a17-0d6b-4822-9b3b-d2ded81dc1c2 admin admin] PATH_INFO: `/v3/projects/02d677c3bdea49bd8454f301e29a2615/users/43ac920dd1cf41a1a18df8b738cc88a6/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:43.018680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa989a17-0d6b-4822-9b3b-d2ded81dc1c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02d677c3bdea49bd8454f301e29a2615, user_id=43ac920dd1cf41a1a18df8b738cc88a6, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:43.021765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa989a17-0d6b-4822-9b3b-d2ded81dc1c2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:43.033545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1039/2076] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:42 2022] PUT /identity/v3/projects/02d677c3bdea49bd8454f301e29a2615/users/43ac920dd1cf41a1a18df8b738cc88a6/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:43.045410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56172afe-842d-4127-b488-ed2facb92e24 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:43.045702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56172afe-842d-4127-b488-ed2facb92e24 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:43.045979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56172afe-842d-4127-b488-ed2facb92e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:43.079075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-56172afe-842d-4127-b488-ed2facb92e24 None None] MFA Rules not processed for user `43ac920dd1cf41a1a18df8b738cc88a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:43.080353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-56172afe-842d-4127-b488-ed2facb92e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:43.176603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1038/2077] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:43.189878 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa989a17-0d6b-4822-9b3b-d2ded81dc1c2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:43.191878 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fa989a17-0d6b-4822-9b3b-d2ded81dc1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:43.273904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:43.285334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa989a17-0d6b-4822-9b3b-d2ded81dc1c2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:43.291726 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135006e5-fbd8-4cd9-b9be-b437c15b8bd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:43.292484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135006e5-fbd8-4cd9-b9be-b437c15b8bd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:43.292834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135006e5-fbd8-4cd9-b9be-b437c15b8bd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:43.293158 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135006e5-fbd8-4cd9-b9be-b437c15b8bd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:43.308608 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135006e5-fbd8-4cd9-b9be-b437c15b8bd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:43.311483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135006e5-fbd8-4cd9-b9be-b437c15b8bd6 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:43.328480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1040/2078] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:24:43.479839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56172afe-842d-4127-b488-ed2facb92e24 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:43.503289 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56172afe-842d-4127-b488-ed2facb92e24 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:43.513345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaa838d-8846-4811-9c5e-81494d01769f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:43.514834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa838d-8846-4811-9c5e-81494d01769f service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:43.515335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa838d-8846-4811-9c5e-81494d01769f service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:43.516281 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaa838d-8846-4811-9c5e-81494d01769f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:43.533638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:43.672607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa838d-8846-4811-9c5e-81494d01769f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:43.675515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaa838d-8846-4811-9c5e-81494d01769f service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:43.702320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1039/2079] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:44.337344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135006e5-fbd8-4cd9-b9be-b437c15b8bd6 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:44.429846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135006e5-fbd8-4cd9-b9be-b437c15b8bd6 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:44.438303 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:44.439688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:44.440305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:44.440898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] PATH_INFO: `/v3/users/625e0595518d45d3b564f6d9a1d48cb0` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:44.443765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=625e0595518d45d3b564f6d9a1d48cb0)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:44.445998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:44.539970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '625e0595518d45d3b564f6d9a1d48cb0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:44.550909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '625e0595518d45d3b564f6d9a1d48cb0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:44.551697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '625e0595518d45d3b564f6d9a1d48cb0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:44.562699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '625e0595518d45d3b564f6d9a1d48cb0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:44.577941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '625e0595518d45d3b564f6d9a1d48cb0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:44.588813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1041/2080] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:44 2022] DELETE /identity/v3/users/625e0595518d45d3b564f6d9a1d48cb0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:44.591689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff9fb-2a8b-45ff-89e2-9f15bae41914 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:44.592200 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff9fb-2a8b-45ff-89e2-9f15bae41914 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:44.592692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff9fb-2a8b-45ff-89e2-9f15bae41914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:44.629261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-638ff9fb-2a8b-45ff-89e2-9f15bae41914 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:44.631685 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-638ff9fb-2a8b-45ff-89e2-9f15bae41914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:44.721719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1040/2081] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:44.734180 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:44.735800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:44.805021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69279a9-6a45-4f06-a2ab-bba939f67b4f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:44.809756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0cf3a-c059-4a23-9d7b-6c3be433caef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:44.810503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0cf3a-c059-4a23-9d7b-6c3be433caef service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:44.810898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0cf3a-c059-4a23-9d7b-6c3be433caef service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:44.811236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0cf3a-c059-4a23-9d7b-6c3be433caef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:44.823721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0cf3a-c059-4a23-9d7b-6c3be433caef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:44.825389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0cf3a-c059-4a23-9d7b-6c3be433caef service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:44.839895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1042/2082] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:45.300013 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff9fb-2a8b-45ff-89e2-9f15bae41914 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:45.303580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-638ff9fb-2a8b-45ff-89e2-9f15bae41914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:45.326877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff9fb-2a8b-45ff-89e2-9f15bae41914 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:45.331664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:45.332315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:45.332659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:45.332866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] PATH_INFO: `/v3/projects/8c366b91687f434e944d21904600f567` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:45.334833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c366b91687f434e944d21904600f567)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:45.336781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:45.398264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c366b91687f434e944d21904600f567', 'request_id': 'req-eba37e08-654c-44f4-ab80-de29361b115c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:45.407770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c366b91687f434e944d21904600f567', 'request_id': 'req-eba37e08-654c-44f4-ab80-de29361b115c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:45.415238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c366b91687f434e944d21904600f567', 'request_id': 'req-eba37e08-654c-44f4-ab80-de29361b115c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:45.430112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] The token cache is being invalidate because project 8c366b91687f434e944d21904600f567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:45.430414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:45.431081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:45.432597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1041/2083] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:45 2022] DELETE /identity/v3/projects/8c366b91687f434e944d21904600f567 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:45.437340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0cf3a-c059-4a23-9d7b-6c3be433caef service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:45.456399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0cf3a-c059-4a23-9d7b-6c3be433caef service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:45.473479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:45.553993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df0074-faf6-49a0-8fe7-abec6f0edec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:45.554740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df0074-faf6-49a0-8fe7-abec6f0edec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:45.555294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df0074-faf6-49a0-8fe7-abec6f0edec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:45.555572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df0074-faf6-49a0-8fe7-abec6f0edec8 admin admin] PATH_INFO: `/v3/users/43ac920dd1cf41a1a18df8b738cc88a6` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:45.556889 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df0074-faf6-49a0-8fe7-abec6f0edec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ac920dd1cf41a1a18df8b738cc88a6)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:45.559108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df0074-faf6-49a0-8fe7-abec6f0edec8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:45.640478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f4df0074-faf6-49a0-8fe7-abec6f0edec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ac920dd1cf41a1a18df8b738cc88a6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:45.649975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f4df0074-faf6-49a0-8fe7-abec6f0edec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ac920dd1cf41a1a18df8b738cc88a6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:45.650734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f4df0074-faf6-49a0-8fe7-abec6f0edec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ac920dd1cf41a1a18df8b738cc88a6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:45.659576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f4df0074-faf6-49a0-8fe7-abec6f0edec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ac920dd1cf41a1a18df8b738cc88a6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:45.679940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-f4df0074-faf6-49a0-8fe7-abec6f0edec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ac920dd1cf41a1a18df8b738cc88a6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:45.691721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1043/2084] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:45 2022] DELETE /identity/v3/users/43ac920dd1cf41a1a18df8b738cc88a6 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:46.238950 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:46.240764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:46.266678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba37e08-654c-44f4-ab80-de29361b115c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:46.352945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:46.404840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:46.406008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:46.406570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:46.407317 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244 admin admin] PATH_INFO: `/v3/projects/02d677c3bdea49bd8454f301e29a2615` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:46.409446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02d677c3bdea49bd8454f301e29a2615)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:46.413381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:46.480664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02d677c3bdea49bd8454f301e29a2615', 'request_id': 'req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:46.491104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02d677c3bdea49bd8454f301e29a2615', 'request_id': 'req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:46.498082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02d677c3bdea49bd8454f301e29a2615', 'request_id': 'req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:46.510747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244 admin admin] The token cache is being invalidate because project 02d677c3bdea49bd8454f301e29a2615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:46.511050 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:46.511689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d6a6eef5-4676-4f0b-b50a-e0e893ae9244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:46.513557 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1042/2085] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:46 2022] DELETE /identity/v3/projects/02d677c3bdea49bd8454f301e29a2615 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:46.518474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08f599-4903-4908-a0b1-34012b55e0ac None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:46.519015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08f599-4903-4908-a0b1-34012b55e0ac None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:46.519107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08f599-4903-4908-a0b1-34012b55e0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:46.541106 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2d08f599-4903-4908-a0b1-34012b55e0ac None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:46.542871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2d08f599-4903-4908-a0b1-34012b55e0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:46.654403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1044/2086] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:46.661951 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c10c6c-9e1c-45f5-8038-c62961ea837a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:46.662648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c10c6c-9e1c-45f5-8038-c62961ea837a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:46.663107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c10c6c-9e1c-45f5-8038-c62961ea837a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:46.694304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d3c10c6c-9e1c-45f5-8038-c62961ea837a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:46.696896 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d3c10c6c-9e1c-45f5-8038-c62961ea837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:46.711694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1043/2087] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:46.717635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08f599-4903-4908-a0b1-34012b55e0ac None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:46.736307 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08f599-4903-4908-a0b1-34012b55e0ac None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:46.745553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232ec1d-bc24-4c0f-8b96-31cc400882b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:46.746294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232ec1d-bc24-4c0f-8b96-31cc400882b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:46.746586 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232ec1d-bc24-4c0f-8b96-31cc400882b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:46.747231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232ec1d-bc24-4c0f-8b96-31cc400882b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:46.748084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232ec1d-bc24-4c0f-8b96-31cc400882b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:24:46.748389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232ec1d-bc24-4c0f-8b96-31cc400882b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:46.750492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232ec1d-bc24-4c0f-8b96-31cc400882b0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:46.765646 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1045/2088] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:24:46.769822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c10c6c-9e1c-45f5-8038-c62961ea837a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:46.784163 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c10c6c-9e1c-45f5-8038-c62961ea837a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:46.791061 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d30f9-22cb-4a1e-b289-442938f4c444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:46.791996 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d30f9-22cb-4a1e-b289-442938f4c444 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:46.792679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d30f9-22cb-4a1e-b289-442938f4c444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:46.793089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d30f9-22cb-4a1e-b289-442938f4c444 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:46.793986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d30f9-22cb-4a1e-b289-442938f4c444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:46.797023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d30f9-22cb-4a1e-b289-442938f4c444 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:46.838980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1044/2089] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:46 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:24:46.843165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232ec1d-bc24-4c0f-8b96-31cc400882b0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:46.867738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232ec1d-bc24-4c0f-8b96-31cc400882b0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:46.891605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:46.954132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e89e6-304d-4b22-bd60-354253dbdc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:46.954757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e89e6-304d-4b22-bd60-354253dbdc4b admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:46.955036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e89e6-304d-4b22-bd60-354253dbdc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:46.955310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e89e6-304d-4b22-bd60-354253dbdc4b admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:46.955952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e89e6-304d-4b22-bd60-354253dbdc4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:46.957882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245e89e6-304d-4b22-bd60-354253dbdc4b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:46.986408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1046/2090] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:46 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:24:46.992878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d30f9-22cb-4a1e-b289-442938f4c444 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:47.015986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d30f9-22cb-4a1e-b289-442938f4c444 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:47.034378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e7bed-05c6-4c3a-b6c9-67b0e7a29d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:47.035506 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e7bed-05c6-4c3a-b6c9-67b0e7a29d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:47.035933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e7bed-05c6-4c3a-b6c9-67b0e7a29d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:47.036432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e7bed-05c6-4c3a-b6c9-67b0e7a29d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:47.037620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e7bed-05c6-4c3a-b6c9-67b0e7a29d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:47.040611 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e7bed-05c6-4c3a-b6c9-67b0e7a29d08 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:47.060894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1045/2091] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:46 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:47.065713 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e89e6-304d-4b22-bd60-354253dbdc4b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:47.084139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e89e6-304d-4b22-bd60-354253dbdc4b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:47.089349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58984821-b81c-4817-a92e-10f23bc90edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:47.089950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58984821-b81c-4817-a92e-10f23bc90edb admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:47.090268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58984821-b81c-4817-a92e-10f23bc90edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:47.090547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58984821-b81c-4817-a92e-10f23bc90edb admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:47.091223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58984821-b81c-4817-a92e-10f23bc90edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:47.092893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58984821-b81c-4817-a92e-10f23bc90edb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:47.108407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1047/2092] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:47.113116 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e7bed-05c6-4c3a-b6c9-67b0e7a29d08 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:47.124258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e7bed-05c6-4c3a-b6c9-67b0e7a29d08 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:47.129453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae1a4d-ffc8-4550-9be8-4738d69c6496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:47.130748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae1a4d-ffc8-4550-9be8-4738d69c6496 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:47.131033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae1a4d-ffc8-4550-9be8-4738d69c6496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:47.131349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae1a4d-ffc8-4550-9be8-4738d69c6496 admin admin] PATH_INFO: `/v3/projects/6444dbbebb404dbcb884b97a3770483a/users/8474675ac4b545cf86cce9378eebd75a/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:47.147970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae1a4d-ffc8-4550-9be8-4738d69c6496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6444dbbebb404dbcb884b97a3770483a, user_id=8474675ac4b545cf86cce9378eebd75a, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:47.150705 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ae1a4d-ffc8-4550-9be8-4738d69c6496 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:47.165855 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1046/2093] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:47 2022] PUT /identity/v3/projects/6444dbbebb404dbcb884b97a3770483a/users/8474675ac4b545cf86cce9378eebd75a/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:47.177668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96d26f-ead9-497e-bb8a-413a56164332 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:47.178102 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96d26f-ead9-497e-bb8a-413a56164332 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:47.178495 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec96d26f-ead9-497e-bb8a-413a56164332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:47.221048 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ec96d26f-ead9-497e-bb8a-413a56164332 None None] MFA Rules not processed for user `8474675ac4b545cf86cce9378eebd75a`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:47.222342 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ec96d26f-ead9-497e-bb8a-413a56164332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:47.309564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1048/2094] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:47.323912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae1a4d-ffc8-4550-9be8-4738d69c6496 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:47.326678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-86ae1a4d-ffc8-4550-9be8-4738d69c6496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:47.432155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ae1a4d-ffc8-4550-9be8-4738d69c6496 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:47.436367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155c322-78ab-4412-8026-bcc304cde4d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:47.437073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155c322-78ab-4412-8026-bcc304cde4d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:47.437350 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155c322-78ab-4412-8026-bcc304cde4d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:47.437628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155c322-78ab-4412-8026-bcc304cde4d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:47.448381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155c322-78ab-4412-8026-bcc304cde4d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:47.449853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155c322-78ab-4412-8026-bcc304cde4d2 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:47.460492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1047/2095] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:55.915993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96d26f-ead9-497e-bb8a-413a56164332 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:55.917323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ec96d26f-ead9-497e-bb8a-413a56164332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:56.053894 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec96d26f-ead9-497e-bb8a-413a56164332 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:56.060312 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:56.061137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:56.061537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:56.061958 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] PATH_INFO: `/v3/users/7582a7c5588b42acbdf34b63df997bf8` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:56.065013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7582a7c5588b42acbdf34b63df997bf8)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:56.067416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:56.137385 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:56.182011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7582a7c5588b42acbdf34b63df997bf8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:56.202009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7582a7c5588b42acbdf34b63df997bf8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:56.210452 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7582a7c5588b42acbdf34b63df997bf8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:56.228448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7582a7c5588b42acbdf34b63df997bf8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:56.272024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7582a7c5588b42acbdf34b63df997bf8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:56.283903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1049/2096] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:55 2022] DELETE /identity/v3/users/7582a7c5588b42acbdf34b63df997bf8 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:56.290740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dcfb5-08a0-4836-9e1b-fd7f7bcdec16 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:56.292089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dcfb5-08a0-4836-9e1b-fd7f7bcdec16 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:56.292986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0dcfb5-08a0-4836-9e1b-fd7f7bcdec16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:56.314032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:56.336171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5e0dcfb5-08a0-4836-9e1b-fd7f7bcdec16 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:56.340257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5e0dcfb5-08a0-4836-9e1b-fd7f7bcdec16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:56.489394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1048/2097] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:56.502179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:56.503679 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:56.595711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd32bb-e9dc-4edb-831f-a93bb28e52f0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:56.601319 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106f006-d5a0-43d3-b824-5dacf73634dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:56.601973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106f006-d5a0-43d3-b824-5dacf73634dd service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:56.602282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106f006-d5a0-43d3-b824-5dacf73634dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:56.602548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106f006-d5a0-43d3-b824-5dacf73634dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:56.616780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106f006-d5a0-43d3-b824-5dacf73634dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:56.618661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106f006-d5a0-43d3-b824-5dacf73634dd service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:56.636986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1050/2098] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:24:57.141265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0dcfb5-08a0-4836-9e1b-fd7f7bcdec16 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:57.143670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5e0dcfb5-08a0-4836-9e1b-fd7f7bcdec16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:57.162634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0dcfb5-08a0-4836-9e1b-fd7f7bcdec16 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:57.168824 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:57.169499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:57.169847 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:57.170040 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932 admin admin] PATH_INFO: `/v3/projects/cb83e7659d7e434fb1b4fabc703d0506` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:57.171514 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb83e7659d7e434fb1b4fabc703d0506)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:57.173698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:57.244952 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb83e7659d7e434fb1b4fabc703d0506', 'request_id': 'req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:57.258741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb83e7659d7e434fb1b4fabc703d0506', 'request_id': 'req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:57.270836 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb83e7659d7e434fb1b4fabc703d0506', 'request_id': 'req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:57.294964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932 admin admin] The token cache is being invalidate because project cb83e7659d7e434fb1b4fabc703d0506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:57.295640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:57.296888 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-847d51a8-a5d2-43ad-8bc8-0fd0f41ab932 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:57.300938 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1049/2099] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:57 2022] DELETE /identity/v3/projects/cb83e7659d7e434fb1b4fabc703d0506 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:57.307098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615478c-a311-4297-81ad-d8f015685860 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:57.308040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615478c-a311-4297-81ad-d8f015685860 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:57.308381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2615478c-a311-4297-81ad-d8f015685860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:57.346083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2615478c-a311-4297-81ad-d8f015685860 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:57.348404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2615478c-a311-4297-81ad-d8f015685860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:57.513987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1051/2100] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:57.524518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce30539-6b4b-4229-b8f6-88ee45074ab7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:57.525308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce30539-6b4b-4229-b8f6-88ee45074ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:57.525851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce30539-6b4b-4229-b8f6-88ee45074ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:57.595212 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7ce30539-6b4b-4229-b8f6-88ee45074ab7 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:57.598964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7ce30539-6b4b-4229-b8f6-88ee45074ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:57.617870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1050/2101] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:57.622812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615478c-a311-4297-81ad-d8f015685860 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:57.635832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2615478c-a311-4297-81ad-d8f015685860 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:57.640915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8fc9b-69e4-4e5f-b076-c55efd7a68fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:57.641556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8fc9b-69e4-4e5f-b076-c55efd7a68fe admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:57.641859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8fc9b-69e4-4e5f-b076-c55efd7a68fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:57.642164 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8fc9b-69e4-4e5f-b076-c55efd7a68fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:57.642860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8fc9b-69e4-4e5f-b076-c55efd7a68fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:24:57.643171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8fc9b-69e4-4e5f-b076-c55efd7a68fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:57.645214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8fc9b-69e4-4e5f-b076-c55efd7a68fe admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:57.666618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1052/2102] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:24:57.669788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce30539-6b4b-4229-b8f6-88ee45074ab7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:57.681578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce30539-6b4b-4229-b8f6-88ee45074ab7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:57.688421 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb4c19-1850-435a-afd4-6d2ebeb15161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:57.689099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb4c19-1850-435a-afd4-6d2ebeb15161 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:57.689478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb4c19-1850-435a-afd4-6d2ebeb15161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:57.689667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb4c19-1850-435a-afd4-6d2ebeb15161 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:57.690594 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb4c19-1850-435a-afd4-6d2ebeb15161 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:57.693584 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb4c19-1850-435a-afd4-6d2ebeb15161 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:57.735908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1051/2103] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:24:57 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:24:57.740852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8fc9b-69e4-4e5f-b076-c55efd7a68fe admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:57.756933 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8fc9b-69e4-4e5f-b076-c55efd7a68fe admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:57.821916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:57.875615 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11fa8e-cc9b-49d8-9021-18a5a81553f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:57.876318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11fa8e-cc9b-49d8-9021-18a5a81553f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:57.876704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11fa8e-cc9b-49d8-9021-18a5a81553f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:57.876883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11fa8e-cc9b-49d8-9021-18a5a81553f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:57.877641 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11fa8e-cc9b-49d8-9021-18a5a81553f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:57.881602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11fa8e-cc9b-49d8-9021-18a5a81553f8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:57.927138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1053/2104] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:24:57 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:24:57.935721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb4c19-1850-435a-afd4-6d2ebeb15161 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:57.956666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb4c19-1850-435a-afd4-6d2ebeb15161 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:57.963155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15855d62-67bc-43e1-bc5f-c5132aec5582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:57.963786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15855d62-67bc-43e1-bc5f-c5132aec5582 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:57.964069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15855d62-67bc-43e1-bc5f-c5132aec5582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:57.964343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15855d62-67bc-43e1-bc5f-c5132aec5582 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:57.965008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15855d62-67bc-43e1-bc5f-c5132aec5582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:57.966701 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15855d62-67bc-43e1-bc5f-c5132aec5582 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:57.978894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1052/2105] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:24:57.984314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11fa8e-cc9b-49d8-9021-18a5a81553f8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:58.001848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11fa8e-cc9b-49d8-9021-18a5a81553f8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:58.008933 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899101bd-5a86-4eba-a856-ceb855272fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:58.009662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899101bd-5a86-4eba-a856-ceb855272fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:58.010021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899101bd-5a86-4eba-a856-ceb855272fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:58.011177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899101bd-5a86-4eba-a856-ceb855272fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:58.011953 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899101bd-5a86-4eba-a856-ceb855272fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:58.013892 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899101bd-5a86-4eba-a856-ceb855272fa8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:58.038126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1054/2106] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:24:58.043052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15855d62-67bc-43e1-bc5f-c5132aec5582 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:58.062202 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15855d62-67bc-43e1-bc5f-c5132aec5582 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:58.070839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6ec2-9560-4f11-adbf-d803890e86df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:58.071931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6ec2-9560-4f11-adbf-d803890e86df admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:58.072534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6ec2-9560-4f11-adbf-d803890e86df admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:58.073069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b6ec2-9560-4f11-adbf-d803890e86df admin admin] PATH_INFO: `/v3/projects/acd48c5a19c24554bb48e4aecd5d402c/users/c93b5c6648b842ce9c83117407170286/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:58.102467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b6ec2-9560-4f11-adbf-d803890e86df admin admin] RBAC: Authorizing `identity:create_grant(project_id=acd48c5a19c24554bb48e4aecd5d402c, user_id=c93b5c6648b842ce9c83117407170286, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:58.109742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b6ec2-9560-4f11-adbf-d803890e86df admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:58.133235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1053/2107] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:24:58 2022] PUT /identity/v3/projects/acd48c5a19c24554bb48e4aecd5d402c/users/c93b5c6648b842ce9c83117407170286/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:58.150745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae096f-0216-45d4-b2e2-ea85fab846e7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:58.151370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae096f-0216-45d4-b2e2-ea85fab846e7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:58.151854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae096f-0216-45d4-b2e2-ea85fab846e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:58.194788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-59ae096f-0216-45d4-b2e2-ea85fab846e7 None None] MFA Rules not processed for user `c93b5c6648b842ce9c83117407170286`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:58.197128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-59ae096f-0216-45d4-b2e2-ea85fab846e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:58.307145 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1055/2108] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:58.322548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6ec2-9560-4f11-adbf-d803890e86df admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:58.324515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7c4b6ec2-9560-4f11-adbf-d803890e86df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:58.365325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:58.426120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b6ec2-9560-4f11-adbf-d803890e86df admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:58.430450 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36527ad9-3e43-4678-aa44-ad6aa4600e9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:58.431177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36527ad9-3e43-4678-aa44-ad6aa4600e9f service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:58.431670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36527ad9-3e43-4678-aa44-ad6aa4600e9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:58.432054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36527ad9-3e43-4678-aa44-ad6aa4600e9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:58.442240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36527ad9-3e43-4678-aa44-ad6aa4600e9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:58.443796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36527ad9-3e43-4678-aa44-ad6aa4600e9f service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:58.454741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1054/2109] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:58.490549 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae096f-0216-45d4-b2e2-ea85fab846e7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:58.589690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae096f-0216-45d4-b2e2-ea85fab846e7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:58.594538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:58.595194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:58.595497 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:58.595769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] PATH_INFO: `/v3/users/c93b5c6648b842ce9c83117407170286` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:58.597024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93b5c6648b842ce9c83117407170286)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:58.599709 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:58.654987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:58.657496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93b5c6648b842ce9c83117407170286'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:58.669570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93b5c6648b842ce9c83117407170286'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:58.670652 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93b5c6648b842ce9c83117407170286'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:58.685376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93b5c6648b842ce9c83117407170286'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:58.701090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93b5c6648b842ce9c83117407170286'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:58.709551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1056/2110] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:58 2022] DELETE /identity/v3/users/c93b5c6648b842ce9c83117407170286 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:58.715992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b4580-5ea1-4747-b13e-18926d399a40 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:58.716455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b4580-5ea1-4747-b13e-18926d399a40 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:58.716870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b4580-5ea1-4747-b13e-18926d399a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:58.739918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-142b4580-5ea1-4747-b13e-18926d399a40 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:58.741114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-142b4580-5ea1-4747-b13e-18926d399a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:58.850562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1055/2111] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:58.858743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:58.860686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:58.937294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b4580-5ea1-4747-b13e-18926d399a40 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:58.938190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b495106c-ce2f-42e8-9344-826a2739c52d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:58.938757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-142b4580-5ea1-4747-b13e-18926d399a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:58.943320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec7801-d77a-43cb-83a1-f32a5be34245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:58.944109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec7801-d77a-43cb-83a1-f32a5be34245 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:58.944519 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec7801-d77a-43cb-83a1-f32a5be34245 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:58.944924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec7801-d77a-43cb-83a1-f32a5be34245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:58.953239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b4580-5ea1-4747-b13e-18926d399a40 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:58.958995 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ebf35-87e3-4d6f-b882-af205a194e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:58.959655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ebf35-87e3-4d6f-b882-af205a194e4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:58.959961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ebf35-87e3-4d6f-b882-af205a194e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:58.960321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ebf35-87e3-4d6f-b882-af205a194e4a admin admin] PATH_INFO: `/v3/users/5b8d7610c56147cca292e2552fd25cc9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:58.961505 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ebf35-87e3-4d6f-b882-af205a194e4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b8d7610c56147cca292e2552fd25cc9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:58.963470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ebf35-87e3-4d6f-b882-af205a194e4a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:58.964792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec7801-d77a-43cb-83a1-f32a5be34245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:58.967011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec7801-d77a-43cb-83a1-f32a5be34245 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:58.988391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1057/2112] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:24:58.989576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec7801-d77a-43cb-83a1-f32a5be34245 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:58.991987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a5ec7801-d77a-43cb-83a1-f32a5be34245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:59.012720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec7801-d77a-43cb-83a1-f32a5be34245 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:59.021470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cff87b-e8f6-460a-b532-e8f2d9683df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:59.022417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff87b-e8f6-460a-b532-e8f2d9683df6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:59.023185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff87b-e8f6-460a-b532-e8f2d9683df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:59.023518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff87b-e8f6-460a-b532-e8f2d9683df6 admin admin] PATH_INFO: `/v3/users/8474675ac4b545cf86cce9378eebd75a` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:59.025623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cff87b-e8f6-460a-b532-e8f2d9683df6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8474675ac4b545cf86cce9378eebd75a)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:59.029127 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cff87b-e8f6-460a-b532-e8f2d9683df6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:59.048735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f69ebf35-87e3-4d6f-b882-af205a194e4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b8d7610c56147cca292e2552fd25cc9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.063781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f69ebf35-87e3-4d6f-b882-af205a194e4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b8d7610c56147cca292e2552fd25cc9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.111211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f69ebf35-87e3-4d6f-b882-af205a194e4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b8d7610c56147cca292e2552fd25cc9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.112634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f69ebf35-87e3-4d6f-b882-af205a194e4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b8d7610c56147cca292e2552fd25cc9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.126779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e4cff87b-e8f6-460a-b532-e8f2d9683df6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8474675ac4b545cf86cce9378eebd75a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.128796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f69ebf35-87e3-4d6f-b882-af205a194e4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b8d7610c56147cca292e2552fd25cc9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.140941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e4cff87b-e8f6-460a-b532-e8f2d9683df6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8474675ac4b545cf86cce9378eebd75a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.141254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1056/2113] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:58 2022] DELETE /identity/v3/users/5b8d7610c56147cca292e2552fd25cc9 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:24:59.142872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e4cff87b-e8f6-460a-b532-e8f2d9683df6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8474675ac4b545cf86cce9378eebd75a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.150737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e83753-86fd-431c-9c27-68dc6fe6c4c8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:59.151216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e83753-86fd-431c-9c27-68dc6fe6c4c8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:59.151723 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e83753-86fd-431c-9c27-68dc6fe6c4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:59.156767 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e4cff87b-e8f6-460a-b532-e8f2d9683df6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8474675ac4b545cf86cce9378eebd75a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.173489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-35e83753-86fd-431c-9c27-68dc6fe6c4c8 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:59.174910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-35e83753-86fd-431c-9c27-68dc6fe6c4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:59.185433 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e4cff87b-e8f6-460a-b532-e8f2d9683df6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8474675ac4b545cf86cce9378eebd75a'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.196685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1058/2114] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:24:58 2022] DELETE /identity/v3/users/8474675ac4b545cf86cce9378eebd75a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:59.203245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d02f21-e74c-4f84-bc69-0ae2c7c38042 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:59.203728 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d02f21-e74c-4f84-bc69-0ae2c7c38042 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:59.204664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d02f21-e74c-4f84-bc69-0ae2c7c38042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:59.227328 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:59.230457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e6d02f21-e74c-4f84-bc69-0ae2c7c38042 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:59.231926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e6d02f21-e74c-4f84-bc69-0ae2c7c38042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:59.274429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1057/2115] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:24:59.286187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e83753-86fd-431c-9c27-68dc6fe6c4c8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:59.288749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-35e83753-86fd-431c-9c27-68dc6fe6c4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:59.307257 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:24:59.351811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1059/2116] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:59.364535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d02f21-e74c-4f84-bc69-0ae2c7c38042 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:59.365965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e6d02f21-e74c-4f84-bc69-0ae2c7c38042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:59.389042 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e83753-86fd-431c-9c27-68dc6fe6c4c8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:59.440449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d02f21-e74c-4f84-bc69-0ae2c7c38042 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:59.445563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3003a71-9e62-4c05-aa87-ce448d2b4334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:59.445930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3003a71-9e62-4c05-aa87-ce448d2b4334 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:59.446236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3003a71-9e62-4c05-aa87-ce448d2b4334 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:59.446514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3003a71-9e62-4c05-aa87-ce448d2b4334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:59.459876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3003a71-9e62-4c05-aa87-ce448d2b4334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:59.461864 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3003a71-9e62-4c05-aa87-ce448d2b4334 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:59.477371 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1060/2117] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:24:59.488224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa5181-03d5-4a62-a5dd-4e1a72dceb6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:59.488224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa5181-03d5-4a62-a5dd-4e1a72dceb6f service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:59.488224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa5181-03d5-4a62-a5dd-4e1a72dceb6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:59.488224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa5181-03d5-4a62-a5dd-4e1a72dceb6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:59.490598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3003a71-9e62-4c05-aa87-ce448d2b4334 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:59.492874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d3003a71-9e62-4c05-aa87-ce448d2b4334 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:59.509473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa5181-03d5-4a62-a5dd-4e1a72dceb6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:59.513162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa5181-03d5-4a62-a5dd-4e1a72dceb6f service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:59.526965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3003a71-9e62-4c05-aa87-ce448d2b4334 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:24:59.536808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288e945-4e15-433f-9de1-01ff501663cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:24:59.538343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288e945-4e15-433f-9de1-01ff501663cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:59.538810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288e945-4e15-433f-9de1-01ff501663cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:59.539368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288e945-4e15-433f-9de1-01ff501663cd admin admin] PATH_INFO: `/v3/projects/acd48c5a19c24554bb48e4aecd5d402c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:59.541992 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288e945-4e15-433f-9de1-01ff501663cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=acd48c5a19c24554bb48e4aecd5d402c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:24:59.545697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288e945-4e15-433f-9de1-01ff501663cd admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:24:59.553575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1058/2118] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:24:59.658747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2288e945-4e15-433f-9de1-01ff501663cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acd48c5a19c24554bb48e4aecd5d402c', 'request_id': 'req-2288e945-4e15-433f-9de1-01ff501663cd'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.676614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2288e945-4e15-433f-9de1-01ff501663cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acd48c5a19c24554bb48e4aecd5d402c', 'request_id': 'req-2288e945-4e15-433f-9de1-01ff501663cd'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.690574 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2288e945-4e15-433f-9de1-01ff501663cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acd48c5a19c24554bb48e4aecd5d402c', 'request_id': 'req-2288e945-4e15-433f-9de1-01ff501663cd'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.714512 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2288e945-4e15-433f-9de1-01ff501663cd admin admin] The token cache is being invalidate because project acd48c5a19c24554bb48e4aecd5d402c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:24:59.714965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2288e945-4e15-433f-9de1-01ff501663cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.716587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2288e945-4e15-433f-9de1-01ff501663cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:24:59.719146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1061/2119] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:24:59 2022] DELETE /identity/v3/projects/acd48c5a19c24554bb48e4aecd5d402c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:24:59.726322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f70c13-b9c4-40ff-ba52-987b9c9b7116 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:59.726783 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f70c13-b9c4-40ff-ba52-987b9c9b7116 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:59.727464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f70c13-b9c4-40ff-ba52-987b9c9b7116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:59.766644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-66f70c13-b9c4-40ff-ba52-987b9c9b7116 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:59.768707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-66f70c13-b9c4-40ff-ba52-987b9c9b7116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:59.885592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1059/2120] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:59.898086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582805e-c42b-4a6a-a851-6ace9fbdccde None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:24:59.898438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582805e-c42b-4a6a-a851-6ace9fbdccde None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:24:59.898675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582805e-c42b-4a6a-a851-6ace9fbdccde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:24:59.955335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a582805e-c42b-4a6a-a851-6ace9fbdccde None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:24:59.959589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a582805e-c42b-4a6a-a851-6ace9fbdccde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:24:59.974953 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1062/2121] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:24:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:24:59.979520 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f70c13-b9c4-40ff-ba52-987b9c9b7116 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:24:59.994259 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f70c13-b9c4-40ff-ba52-987b9c9b7116 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:00.001378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5abcf-0029-438f-a051-090f79db3793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:00.001378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5abcf-0029-438f-a051-090f79db3793 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:00.001378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5abcf-0029-438f-a051-090f79db3793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:00.001378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5abcf-0029-438f-a051-090f79db3793 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:00.002127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5abcf-0029-438f-a051-090f79db3793 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:25:00.002127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5abcf-0029-438f-a051-090f79db3793 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:00.004238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc5abcf-0029-438f-a051-090f79db3793 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:00.018781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1060/2122] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:24:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:25:00.024677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582805e-c42b-4a6a-a851-6ace9fbdccde None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:00.041339 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582805e-c42b-4a6a-a851-6ace9fbdccde None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:00.047550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a3229-5c13-4990-96f2-d9b409c9d7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:00.048331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a3229-5c13-4990-96f2-d9b409c9d7cb admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:00.048705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a3229-5c13-4990-96f2-d9b409c9d7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:00.049076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a3229-5c13-4990-96f2-d9b409c9d7cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:00.049905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a3229-5c13-4990-96f2-d9b409c9d7cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:00.052156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a3229-5c13-4990-96f2-d9b409c9d7cb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:00.089208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1063/2123] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:25:00 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:00.093924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5abcf-0029-438f-a051-090f79db3793 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:00.114010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5abcf-0029-438f-a051-090f79db3793 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:00.188476 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:00.251323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aba854-42ef-4e6c-9612-1bf7d2288f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:00.252601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aba854-42ef-4e6c-9612-1bf7d2288f44 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:00.254773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aba854-42ef-4e6c-9612-1bf7d2288f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:00.255193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aba854-42ef-4e6c-9612-1bf7d2288f44 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:00.256404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aba854-42ef-4e6c-9612-1bf7d2288f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:00.260685 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aba854-42ef-4e6c-9612-1bf7d2288f44 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:00.305287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1061/2124] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:25:00 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:00.310481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a3229-5c13-4990-96f2-d9b409c9d7cb admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:00.323971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a3229-5c13-4990-96f2-d9b409c9d7cb admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:00.329926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8a27c-5e7f-45c9-b3b5-5d5ddbabb57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:00.330923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8a27c-5e7f-45c9-b3b5-5d5ddbabb57b admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:00.331565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8a27c-5e7f-45c9-b3b5-5d5ddbabb57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:00.333453 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8a27c-5e7f-45c9-b3b5-5d5ddbabb57b admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:00.333453 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8a27c-5e7f-45c9-b3b5-5d5ddbabb57b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:00.335773 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c8a27c-5e7f-45c9-b3b5-5d5ddbabb57b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:00.352999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1064/2125] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:25:00.355559 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aba854-42ef-4e6c-9612-1bf7d2288f44 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:00.368300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aba854-42ef-4e6c-9612-1bf7d2288f44 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:00.375297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca67c79-9f37-46bf-9551-f32d1eda5185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:00.376248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca67c79-9f37-46bf-9551-f32d1eda5185 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:00.377311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca67c79-9f37-46bf-9551-f32d1eda5185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:00.377690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca67c79-9f37-46bf-9551-f32d1eda5185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:00.378527 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca67c79-9f37-46bf-9551-f32d1eda5185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:00.381258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca67c79-9f37-46bf-9551-f32d1eda5185 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:00.398479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1062/2126] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:25:00.404600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8a27c-5e7f-45c9-b3b5-5d5ddbabb57b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:00.419800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c8a27c-5e7f-45c9-b3b5-5d5ddbabb57b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:00.446166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756c629-ffd2-42ca-bfa4-f140deb25a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:00.447258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756c629-ffd2-42ca-bfa4-f140deb25a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:00.447848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756c629-ffd2-42ca-bfa4-f140deb25a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:00.448203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756c629-ffd2-42ca-bfa4-f140deb25a6c admin admin] PATH_INFO: `/v3/projects/de5ebe21fe434bc3ad5ab4354c52fcdb/users/22d1610491ba481e80f3e8cccfea3aa1/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:00.478716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756c629-ffd2-42ca-bfa4-f140deb25a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=de5ebe21fe434bc3ad5ab4354c52fcdb, user_id=22d1610491ba481e80f3e8cccfea3aa1, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:00.482181 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3756c629-ffd2-42ca-bfa4-f140deb25a6c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:00.495024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:00.510132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1065/2127] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:25:00 2022] PUT /identity/v3/projects/de5ebe21fe434bc3ad5ab4354c52fcdb/users/22d1610491ba481e80f3e8cccfea3aa1/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:00.510481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca67c79-9f37-46bf-9551-f32d1eda5185 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:00.510481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cca67c79-9f37-46bf-9551-f32d1eda5185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:00.522875 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d8d73-063f-4820-86cd-171408b5b2ca None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:00.523421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d8d73-063f-4820-86cd-171408b5b2ca None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:00.523916 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d8d73-063f-4820-86cd-171408b5b2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:00.536062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca67c79-9f37-46bf-9551-f32d1eda5185 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:00.601137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-494d8d73-063f-4820-86cd-171408b5b2ca None None] MFA Rules not processed for user `22d1610491ba481e80f3e8cccfea3aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:00.602716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-494d8d73-063f-4820-86cd-171408b5b2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:00.658081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:00.659508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:00.660253 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:00.660811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] PATH_INFO: `/v3/projects/6444dbbebb404dbcb884b97a3770483a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:00.663481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6444dbbebb404dbcb884b97a3770483a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:00.667574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:00.738155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1066/2128] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:00.740173 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d8d73-063f-4820-86cd-171408b5b2ca None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:00.745989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-494d8d73-063f-4820-86cd-171408b5b2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:00.756986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6444dbbebb404dbcb884b97a3770483a', 'request_id': 'req-6b448730-45ed-459a-926d-85e7b2a381f5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:00.775199 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494d8d73-063f-4820-86cd-171408b5b2ca None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:00.780521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6444dbbebb404dbcb884b97a3770483a', 'request_id': 'req-6b448730-45ed-459a-926d-85e7b2a381f5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:00.795160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6444dbbebb404dbcb884b97a3770483a', 'request_id': 'req-6b448730-45ed-459a-926d-85e7b2a381f5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:00.828955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] The token cache is being invalidate because project 6444dbbebb404dbcb884b97a3770483a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:25:00.829695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:00.831384 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:00.835059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1063/2129] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:25:00 2022] DELETE /identity/v3/projects/6444dbbebb404dbcb884b97a3770483a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:00.836578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:00.840063 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:00.875530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517177cc-0e1f-4570-98d6-561ede389a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:00.876686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517177cc-0e1f-4570-98d6-561ede389a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:00.877170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517177cc-0e1f-4570-98d6-561ede389a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:00.878264 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517177cc-0e1f-4570-98d6-561ede389a1b admin admin] PATH_INFO: `/v3/projects/0c4953f71d154923b3181877dbf7e195` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:00.881202 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517177cc-0e1f-4570-98d6-561ede389a1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4953f71d154923b3181877dbf7e195)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:00.885675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517177cc-0e1f-4570-98d6-561ede389a1b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:00.950108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:00.986924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-517177cc-0e1f-4570-98d6-561ede389a1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4953f71d154923b3181877dbf7e195', 'request_id': 'req-517177cc-0e1f-4570-98d6-561ede389a1b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:00.987373 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b448730-45ed-459a-926d-85e7b2a381f5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:00.998023 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-517177cc-0e1f-4570-98d6-561ede389a1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4953f71d154923b3181877dbf7e195', 'request_id': 'req-517177cc-0e1f-4570-98d6-561ede389a1b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:01.008174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-517177cc-0e1f-4570-98d6-561ede389a1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4953f71d154923b3181877dbf7e195', 'request_id': 'req-517177cc-0e1f-4570-98d6-561ede389a1b'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:01.025981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-517177cc-0e1f-4570-98d6-561ede389a1b admin admin] The token cache is being invalidate because project 0c4953f71d154923b3181877dbf7e195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:25:01.026359 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-517177cc-0e1f-4570-98d6-561ede389a1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:01.027145 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-517177cc-0e1f-4570-98d6-561ede389a1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:01.029774 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1067/2130] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:25:00 2022] DELETE /identity/v3/projects/0c4953f71d154923b3181877dbf7e195 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:01.032233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2f11c-3a0c-400b-a7ab-f310e4088e02 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.032525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2f11c-3a0c-400b-a7ab-f310e4088e02 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.032954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd2f11c-3a0c-400b-a7ab-f310e4088e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.064782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8bd2f11c-3a0c-400b-a7ab-f310e4088e02 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:01.067407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8bd2f11c-3a0c-400b-a7ab-f310e4088e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:01.097342 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:01.110525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ef796-29d5-443c-a1f0-ee42e69e24fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.112027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ef796-29d5-443c-a1f0-ee42e69e24fa service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.112589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ef796-29d5-443c-a1f0-ee42e69e24fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.113086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ef796-29d5-443c-a1f0-ee42e69e24fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.116536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-637ef796-29d5-443c-a1f0-ee42e69e24fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:01.197687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1068/2131] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:01.199787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8b5dd-c84c-4372-9179-6338dfe0a265 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.200391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8b5dd-c84c-4372-9179-6338dfe0a265 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.200668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8b5dd-c84c-4372-9179-6338dfe0a265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.229005 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7eb8b5dd-c84c-4372-9179-6338dfe0a265 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:01.229324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ef796-29d5-443c-a1f0-ee42e69e24fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.230622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7eb8b5dd-c84c-4372-9179-6338dfe0a265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:01.231591 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ef796-29d5-443c-a1f0-ee42e69e24fa service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.250182 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1069/2132] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:01.250182 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcabc8-b54f-4bfb-8193-691ed80be9fc None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.250182 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcabc8-b54f-4bfb-8193-691ed80be9fc None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.250182 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcabc8-b54f-4bfb-8193-691ed80be9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.250913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1064/2133] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:25:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:25:01.258358 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeac0dd-4091-4b42-a628-2323f6a38882 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.259234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeac0dd-4091-4b42-a628-2323f6a38882 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.261249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeac0dd-4091-4b42-a628-2323f6a38882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.276757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1bdcabc8-b54f-4bfb-8193-691ed80be9fc None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:01.278368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1bdcabc8-b54f-4bfb-8193-691ed80be9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:01.289976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1070/2134] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:01.294544 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcabc8-b54f-4bfb-8193-691ed80be9fc None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:01.304900 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fdeac0dd-4091-4b42-a628-2323f6a38882 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:01.306583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fdeac0dd-4091-4b42-a628-2323f6a38882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:01.307223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcabc8-b54f-4bfb-8193-691ed80be9fc None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:01.315130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e589f-204e-4d20-91b6-85f82e980759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.317111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1065/2135] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:01.318280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e589f-204e-4d20-91b6-85f82e980759 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.318730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e589f-204e-4d20-91b6-85f82e980759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.319146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e589f-204e-4d20-91b6-85f82e980759 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.320545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e589f-204e-4d20-91b6-85f82e980759 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:25:01.321131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeac0dd-4091-4b42-a628-2323f6a38882 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:01.321612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e589f-204e-4d20-91b6-85f82e980759 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.323863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e589f-204e-4d20-91b6-85f82e980759 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.335970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeac0dd-4091-4b42-a628-2323f6a38882 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:01.342251 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1071/2136] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:25:01.344484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484e115-e03d-4f91-9314-885dba936f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.345911 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484e115-e03d-4f91-9314-885dba936f76 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.346472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e589f-204e-4d20-91b6-85f82e980759 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:01.346862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484e115-e03d-4f91-9314-885dba936f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.347641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484e115-e03d-4f91-9314-885dba936f76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.349439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484e115-e03d-4f91-9314-885dba936f76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:25:01.351446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484e115-e03d-4f91-9314-885dba936f76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.355631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484e115-e03d-4f91-9314-885dba936f76 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.358290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e589f-204e-4d20-91b6-85f82e980759 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:01.367618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2c501-34bb-4501-ab50-4cb145f8ff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.367618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2c501-34bb-4501-ab50-4cb145f8ff89 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.367618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2c501-34bb-4501-ab50-4cb145f8ff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.368252 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2c501-34bb-4501-ab50-4cb145f8ff89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.368886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2c501-34bb-4501-ab50-4cb145f8ff89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.372433 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2c501-34bb-4501-ab50-4cb145f8ff89 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.386456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1066/2137] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:25:01.389896 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484e115-e03d-4f91-9314-885dba936f76 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:01.407206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484e115-e03d-4f91-9314-885dba936f76 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:01.414104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195801b-1110-46e4-be28-a97e18e2f1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.414726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195801b-1110-46e4-be28-a97e18e2f1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.415010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195801b-1110-46e4-be28-a97e18e2f1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.415286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195801b-1110-46e4-be28-a97e18e2f1b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.415945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195801b-1110-46e4-be28-a97e18e2f1b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.417969 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195801b-1110-46e4-be28-a97e18e2f1b6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.440216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1072/2138] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:25:01 2022] POST /identity/v3/projects => generated 375 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:01.448343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2c501-34bb-4501-ab50-4cb145f8ff89 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:01.474406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1067/2139] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:25:01 2022] POST /identity/v3/projects => generated 369 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:01.483075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195801b-1110-46e4-be28-a97e18e2f1b6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:01.565291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2c501-34bb-4501-ab50-4cb145f8ff89 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:01.661728 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195801b-1110-46e4-be28-a97e18e2f1b6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:01.662019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d52e6a-9c27-4753-a571-0bfbcbacb6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.662019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d52e6a-9c27-4753-a571-0bfbcbacb6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.662019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d52e6a-9c27-4753-a571-0bfbcbacb6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.662019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d52e6a-9c27-4753-a571-0bfbcbacb6f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.662668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d52e6a-9c27-4753-a571-0bfbcbacb6f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.663455 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d52e6a-9c27-4753-a571-0bfbcbacb6f6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.672649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d21241-84b6-4256-a97c-e1d81786b429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.673165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d21241-84b6-4256-a97c-e1d81786b429 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.673565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d21241-84b6-4256-a97c-e1d81786b429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.674895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d21241-84b6-4256-a97c-e1d81786b429 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.677475 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d21241-84b6-4256-a97c-e1d81786b429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.683764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d21241-84b6-4256-a97c-e1d81786b429 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.698880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:01.716095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1068/2140] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:25:01 2022] POST /identity/v3/users => generated 285 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:01.721344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1073/2141] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:25:01 2022] POST /identity/v3/users => generated 288 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:25:01.722784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d21241-84b6-4256-a97c-e1d81786b429 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:01.731502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d52e6a-9c27-4753-a571-0bfbcbacb6f6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:01.736115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d21241-84b6-4256-a97c-e1d81786b429 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:01.743375 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3dd7a-14ea-4239-b7cf-46032a81483f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.744128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3dd7a-14ea-4239-b7cf-46032a81483f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.744525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3dd7a-14ea-4239-b7cf-46032a81483f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.744946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c3dd7a-14ea-4239-b7cf-46032a81483f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.745640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3dd7a-14ea-4239-b7cf-46032a81483f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.747297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d52e6a-9c27-4753-a571-0bfbcbacb6f6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:01.747889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c3dd7a-14ea-4239-b7cf-46032a81483f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.756210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b669a-7ac0-4500-b171-6f3ddcc9839d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.757002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b669a-7ac0-4500-b171-6f3ddcc9839d admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.757344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b669a-7ac0-4500-b171-6f3ddcc9839d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.757689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b669a-7ac0-4500-b171-6f3ddcc9839d admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.758693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b669a-7ac0-4500-b171-6f3ddcc9839d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.760445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b669a-7ac0-4500-b171-6f3ddcc9839d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.762456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1069/2142] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:25:01.768134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:01.769967 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3dd7a-14ea-4239-b7cf-46032a81483f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:01.779846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1074/2143] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:25:01.781211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b669a-7ac0-4500-b171-6f3ddcc9839d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:01.783779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c3dd7a-14ea-4239-b7cf-46032a81483f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:01.794086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47440696-5e2a-468e-a49a-abf7e5ea1151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.795244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47440696-5e2a-468e-a49a-abf7e5ea1151 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.795525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b669a-7ac0-4500-b171-6f3ddcc9839d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:01.795743 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47440696-5e2a-468e-a49a-abf7e5ea1151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.796145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47440696-5e2a-468e-a49a-abf7e5ea1151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.797889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47440696-5e2a-468e-a49a-abf7e5ea1151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.801995 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47440696-5e2a-468e-a49a-abf7e5ea1151 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.802721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695e637-4fd3-40e9-8eb6-805bca240372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.803679 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695e637-4fd3-40e9-8eb6-805bca240372 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.803924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695e637-4fd3-40e9-8eb6-805bca240372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.804272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695e637-4fd3-40e9-8eb6-805bca240372 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.805024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695e637-4fd3-40e9-8eb6-805bca240372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.807010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695e637-4fd3-40e9-8eb6-805bca240372 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.819095 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1075/2144] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:25:01.823717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695e637-4fd3-40e9-8eb6-805bca240372 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:01.831794 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1070/2145] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:25:01.836175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47440696-5e2a-468e-a49a-abf7e5ea1151 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:01.837323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695e637-4fd3-40e9-8eb6-805bca240372 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:01.842616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b98aa2-c073-49b4-976f-e07beb57b182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.843801 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b98aa2-c073-49b4-976f-e07beb57b182 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.844187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b98aa2-c073-49b4-976f-e07beb57b182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.844572 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b98aa2-c073-49b4-976f-e07beb57b182 admin admin] PATH_INFO: `/v3/projects/6813a2df46ee49ad99ee714f811c6ec3/users/6303820e4a344a9982a0ffc0e8cfaf19/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.851217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47440696-5e2a-468e-a49a-abf7e5ea1151 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:01.857330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b97991-8591-400c-b4b7-b303c044b15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:01.858029 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b97991-8591-400c-b4b7-b303c044b15b admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.858364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b97991-8591-400c-b4b7-b303c044b15b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.858663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b97991-8591-400c-b4b7-b303c044b15b admin admin] PATH_INFO: `/v3/projects/41ad661551c2451b87557b0b97491df6/users/35d1637fd36d42e78c733ce49a2ce845/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.876149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b98aa2-c073-49b4-976f-e07beb57b182 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6813a2df46ee49ad99ee714f811c6ec3, user_id=6303820e4a344a9982a0ffc0e8cfaf19, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.883473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b97991-8591-400c-b4b7-b303c044b15b admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ad661551c2451b87557b0b97491df6, user_id=35d1637fd36d42e78c733ce49a2ce845, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:01.883836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b98aa2-c073-49b4-976f-e07beb57b182 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.886423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b97991-8591-400c-b4b7-b303c044b15b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:01.909310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1071/2146] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:25:01 2022] PUT /identity/v3/projects/41ad661551c2451b87557b0b97491df6/users/35d1637fd36d42e78c733ce49a2ce845/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:25:01.921494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1076/2147] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:25:01 2022] PUT /identity/v3/projects/6813a2df46ee49ad99ee714f811c6ec3/users/6303820e4a344a9982a0ffc0e8cfaf19/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:25:01.922524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536350d-041b-4a05-8d3a-b16a2933a21f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.922524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536350d-041b-4a05-8d3a-b16a2933a21f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.922524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0536350d-041b-4a05-8d3a-b16a2933a21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.934011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186842e-e349-4804-aa80-f638babc8a15 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:01.934463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186842e-e349-4804-aa80-f638babc8a15 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:01.934741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186842e-e349-4804-aa80-f638babc8a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:01.969947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0536350d-041b-4a05-8d3a-b16a2933a21f None None] MFA Rules not processed for user `35d1637fd36d42e78c733ce49a2ce845`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:01.972202 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0536350d-041b-4a05-8d3a-b16a2933a21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:01.996443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3186842e-e349-4804-aa80-f638babc8a15 None None] MFA Rules not processed for user `6303820e4a344a9982a0ffc0e8cfaf19`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:01.998174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3186842e-e349-4804-aa80-f638babc8a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:02.120554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1072/2148] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:02.132004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1077/2149] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:02.139342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536350d-041b-4a05-8d3a-b16a2933a21f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:02.140065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186842e-e349-4804-aa80-f638babc8a15 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:02.142723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3186842e-e349-4804-aa80-f638babc8a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:02.251097 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0536350d-041b-4a05-8d3a-b16a2933a21f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:02.261146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211072d-8c05-4957-a55d-fa022e46b6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:02.262722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211072d-8c05-4957-a55d-fa022e46b6e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:02.263390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211072d-8c05-4957-a55d-fa022e46b6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:02.263729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211072d-8c05-4957-a55d-fa022e46b6e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:02.266345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211072d-8c05-4957-a55d-fa022e46b6e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:02.271287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211072d-8c05-4957-a55d-fa022e46b6e8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:02.289687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186842e-e349-4804-aa80-f638babc8a15 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:02.299172 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fabde-1463-467b-b2f5-997eb816d5db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2d76fb9cbdd40f9b2a748209cee5b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:02.300117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fabde-1463-467b-b2f5-997eb816d5db service glance] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:02.300404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fabde-1463-467b-b2f5-997eb816d5db service glance] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:02.301192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fabde-1463-467b-b2f5-997eb816d5db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:02.313322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fabde-1463-467b-b2f5-997eb816d5db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:02.315281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fabde-1463-467b-b2f5-997eb816d5db service glance] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:02.325214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1073/2150] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:25:02 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:25:02.326478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211072d-8c05-4957-a55d-fa022e46b6e8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:02.329560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1078/2151] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:25:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:25:02.359931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:02.406202 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211072d-8c05-4957-a55d-fa022e46b6e8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:02.413355 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da294190-ea3f-4d14-9ca4-e77f479d1fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:02.414566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da294190-ea3f-4d14-9ca4-e77f479d1fac admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:02.415019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da294190-ea3f-4d14-9ca4-e77f479d1fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:02.415481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da294190-ea3f-4d14-9ca4-e77f479d1fac admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:02.416770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da294190-ea3f-4d14-9ca4-e77f479d1fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:02.420802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da294190-ea3f-4d14-9ca4-e77f479d1fac admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:02.450187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1074/2152] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:25:02 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:02.455757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fabde-1463-467b-b2f5-997eb816d5db service glance] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:02.475066 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fabde-1463-467b-b2f5-997eb816d5db service glance] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:02.490781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cc970-abd4-4027-8da1-7026ac693fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:02.491501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cc970-abd4-4027-8da1-7026ac693fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:02.491873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cc970-abd4-4027-8da1-7026ac693fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:02.492050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cc970-abd4-4027-8da1-7026ac693fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:02.492716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cc970-abd4-4027-8da1-7026ac693fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:02.494412 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cc970-abd4-4027-8da1-7026ac693fb6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:02.513222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1079/2153] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:25:02.513409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da294190-ea3f-4d14-9ca4-e77f479d1fac admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:02.531749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da294190-ea3f-4d14-9ca4-e77f479d1fac admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:02.540975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc819dc-322e-47eb-8182-8c866d580a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:02.542285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc819dc-322e-47eb-8182-8c866d580a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:02.544502 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc819dc-322e-47eb-8182-8c866d580a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:02.545178 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc819dc-322e-47eb-8182-8c866d580a79 admin admin] PATH_INFO: `/v3/projects/72b3ae2eaa3c4a16898b763255335503/users/a4018030eee94cd8819d7cb391b172c6/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:02.567413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc819dc-322e-47eb-8182-8c866d580a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72b3ae2eaa3c4a16898b763255335503, user_id=a4018030eee94cd8819d7cb391b172c6, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:02.570153 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc819dc-322e-47eb-8182-8c866d580a79 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:02.590013 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1075/2154] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:25:02 2022] PUT /identity/v3/projects/72b3ae2eaa3c4a16898b763255335503/users/a4018030eee94cd8819d7cb391b172c6/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:02.596059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cc970-abd4-4027-8da1-7026ac693fb6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:02.608967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cc970-abd4-4027-8da1-7026ac693fb6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:02.643848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:02.695633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a943d5b-8776-4c33-b5c4-ef0f951c977c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:02.696347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a943d5b-8776-4c33-b5c4-ef0f951c977c admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:02.697807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a943d5b-8776-4c33-b5c4-ef0f951c977c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:02.697807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a943d5b-8776-4c33-b5c4-ef0f951c977c admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:02.698682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a943d5b-8776-4c33-b5c4-ef0f951c977c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:02.700636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a943d5b-8776-4c33-b5c4-ef0f951c977c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:02.716502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1080/2155] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:25:02.722034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc819dc-322e-47eb-8182-8c866d580a79 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:02.741373 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc819dc-322e-47eb-8182-8c866d580a79 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:02.749916 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ae32b-7203-4957-bf21-f90cddedb408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:02.751275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ae32b-7203-4957-bf21-f90cddedb408 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:02.751727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ae32b-7203-4957-bf21-f90cddedb408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:02.752180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ae32b-7203-4957-bf21-f90cddedb408 admin admin] PATH_INFO: `/v3/projects/72b3ae2eaa3c4a16898b763255335503/users/a4018030eee94cd8819d7cb391b172c6/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:02.755715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ae32b-7203-4957-bf21-f90cddedb408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72b3ae2eaa3c4a16898b763255335503, user_id=a4018030eee94cd8819d7cb391b172c6, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:02.761267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ae32b-7203-4957-bf21-f90cddedb408 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:02.789718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1076/2156] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:25:02 2022] PUT /identity/v3/projects/72b3ae2eaa3c4a16898b763255335503/users/a4018030eee94cd8819d7cb391b172c6/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:02.804405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948675f3-518b-46ac-8658-f52458edcb92 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:02.806009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948675f3-518b-46ac-8658-f52458edcb92 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:02.806390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948675f3-518b-46ac-8658-f52458edcb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:02.858622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e8d0c-7ba0-4683-ac3e-16295d291dc7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:02.859363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e8d0c-7ba0-4683-ac3e-16295d291dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:02.860117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e8d0c-7ba0-4683-ac3e-16295d291dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:02.868799 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-948675f3-518b-46ac-8658-f52458edcb92 None None] MFA Rules not processed for user `a4018030eee94cd8819d7cb391b172c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:02.870752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-948675f3-518b-46ac-8658-f52458edcb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:02.923151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7f0e8d0c-7ba0-4683-ac3e-16295d291dc7 None None] MFA Rules not processed for user `415e8845ba8d46cc9dc939ef4556e865`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:02.927045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7f0e8d0c-7ba0-4683-ac3e-16295d291dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:03.168818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1081/2157] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:25:03.175390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1077/2158] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:25:03.175781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545999d-4b93-40ca-b351-deaa8fd07898 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:03.175781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545999d-4b93-40ca-b351-deaa8fd07898 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:03.175781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545999d-4b93-40ca-b351-deaa8fd07898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:03.195203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e8d0c-7ba0-4683-ac3e-16295d291dc7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:03.196894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7f0e8d0c-7ba0-4683-ac3e-16295d291dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:03.240935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a545999d-4b93-40ca-b351-deaa8fd07898 None None] MFA Rules not processed for user `22d1610491ba481e80f3e8cccfea3aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:03.243904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a545999d-4b93-40ca-b351-deaa8fd07898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:03.366611 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0e8d0c-7ba0-4683-ac3e-16295d291dc7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:03.373448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa705a-3448-4785-8287-8b33ec1dfa9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:03.374629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa705a-3448-4785-8287-8b33ec1dfa9c service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:03.375114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa705a-3448-4785-8287-8b33ec1dfa9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:03.375557 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa705a-3448-4785-8287-8b33ec1dfa9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:03.381625 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1082/2159] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:03.448601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:03.530479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa705a-3448-4785-8287-8b33ec1dfa9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:03.533032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fa705a-3448-4785-8287-8b33ec1dfa9c service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:03.559334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1078/2160] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:25:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:25:06.859338 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545999d-4b93-40ca-b351-deaa8fd07898 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:06.944329 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:06.961749 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545999d-4b93-40ca-b351-deaa8fd07898 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:06.968239 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:06.968900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:06.969207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:06.969508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] PATH_INFO: `/v3/users/6303820e4a344a9982a0ffc0e8cfaf19` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:06.970756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] RBAC: Authorizing `identity:delete_user(user_id=6303820e4a344a9982a0ffc0e8cfaf19)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:06.973860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:07.045093 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6303820e4a344a9982a0ffc0e8cfaf19'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:07.055557 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6303820e4a344a9982a0ffc0e8cfaf19'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:07.056332 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6303820e4a344a9982a0ffc0e8cfaf19'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:07.066739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6303820e4a344a9982a0ffc0e8cfaf19'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:07.095976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6303820e4a344a9982a0ffc0e8cfaf19'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:07.105531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1083/2161] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:25:06 2022] DELETE /identity/v3/users/6303820e4a344a9982a0ffc0e8cfaf19 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:07.118702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb29f9-e7c2-4655-b44e-70a447247fa6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:07.119667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb29f9-e7c2-4655-b44e-70a447247fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:07.119667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb29f9-e7c2-4655-b44e-70a447247fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:07.164560 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-eaeb29f9-e7c2-4655-b44e-70a447247fa6 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:07.166872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-eaeb29f9-e7c2-4655-b44e-70a447247fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:07.327871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1079/2162] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:07.340518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:07.342148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:07.436269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad332a44-626c-4117-a5a6-e8c49b2d17da admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:07.442478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b74f6f-3b7b-4aa5-b712-96719d8be6ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:07.443589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b74f6f-3b7b-4aa5-b712-96719d8be6ff service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:07.444196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b74f6f-3b7b-4aa5-b712-96719d8be6ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:07.444473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b74f6f-3b7b-4aa5-b712-96719d8be6ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:07.463034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b74f6f-3b7b-4aa5-b712-96719d8be6ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:07.465431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b74f6f-3b7b-4aa5-b712-96719d8be6ff service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:07.510259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1084/2163] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:25:08.070600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb29f9-e7c2-4655-b44e-70a447247fa6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:08.072533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-eaeb29f9-e7c2-4655-b44e-70a447247fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:08.093711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb29f9-e7c2-4655-b44e-70a447247fa6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:08.108130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:08.108805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:08.109484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:08.109484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] PATH_INFO: `/v3/projects/6813a2df46ee49ad99ee714f811c6ec3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:08.111175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6813a2df46ee49ad99ee714f811c6ec3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:08.114224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:08.197388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6813a2df46ee49ad99ee714f811c6ec3', 'request_id': 'req-d7d42caf-d4db-4e13-81a1-eed5672caa09'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:08.213849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6813a2df46ee49ad99ee714f811c6ec3', 'request_id': 'req-d7d42caf-d4db-4e13-81a1-eed5672caa09'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:08.224068 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6813a2df46ee49ad99ee714f811c6ec3', 'request_id': 'req-d7d42caf-d4db-4e13-81a1-eed5672caa09'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:08.248711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] The token cache is being invalidate because project 6813a2df46ee49ad99ee714f811c6ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:25:08.248711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:08.250446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:08.252346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1080/2164] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:25:08 2022] DELETE /identity/v3/projects/6813a2df46ee49ad99ee714f811c6ec3 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:25:08.259381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b74f6f-3b7b-4aa5-b712-96719d8be6ff service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:08.280578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b74f6f-3b7b-4aa5-b712-96719d8be6ff service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:08.393417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81882d18-108a-4f84-a680-005c8ffbe00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:08.394597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81882d18-108a-4f84-a680-005c8ffbe00f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:08.395066 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81882d18-108a-4f84-a680-005c8ffbe00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:08.395527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81882d18-108a-4f84-a680-005c8ffbe00f admin admin] PATH_INFO: `/v3/users/a4018030eee94cd8819d7cb391b172c6` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:08.401012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81882d18-108a-4f84-a680-005c8ffbe00f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4018030eee94cd8819d7cb391b172c6)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:08.406122 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81882d18-108a-4f84-a680-005c8ffbe00f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:08.445477 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:08.543579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-81882d18-108a-4f84-a680-005c8ffbe00f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4018030eee94cd8819d7cb391b172c6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:08.554514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-81882d18-108a-4f84-a680-005c8ffbe00f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4018030eee94cd8819d7cb391b172c6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:08.557376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-81882d18-108a-4f84-a680-005c8ffbe00f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4018030eee94cd8819d7cb391b172c6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:08.590556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-81882d18-108a-4f84-a680-005c8ffbe00f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4018030eee94cd8819d7cb391b172c6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:08.621879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-81882d18-108a-4f84-a680-005c8ffbe00f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4018030eee94cd8819d7cb391b172c6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:08.636881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1085/2165] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:25:08 2022] DELETE /identity/v3/users/a4018030eee94cd8819d7cb391b172c6 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:09.134342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:09.136819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:09.187046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d42caf-d4db-4e13-81a1-eed5672caa09 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:09.307954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:09.320278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915657d-bc1f-4e61-804d-1cb381ddafa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:09.321405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915657d-bc1f-4e61-804d-1cb381ddafa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:09.332891 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915657d-bc1f-4e61-804d-1cb381ddafa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:09.333715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915657d-bc1f-4e61-804d-1cb381ddafa8 admin admin] PATH_INFO: `/v3/projects/72b3ae2eaa3c4a16898b763255335503` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:09.336102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915657d-bc1f-4e61-804d-1cb381ddafa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72b3ae2eaa3c4a16898b763255335503)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:09.338976 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915657d-bc1f-4e61-804d-1cb381ddafa8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:09.455271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1915657d-bc1f-4e61-804d-1cb381ddafa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72b3ae2eaa3c4a16898b763255335503', 'request_id': 'req-1915657d-bc1f-4e61-804d-1cb381ddafa8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:09.472645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1915657d-bc1f-4e61-804d-1cb381ddafa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72b3ae2eaa3c4a16898b763255335503', 'request_id': 'req-1915657d-bc1f-4e61-804d-1cb381ddafa8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:09.486032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1915657d-bc1f-4e61-804d-1cb381ddafa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72b3ae2eaa3c4a16898b763255335503', 'request_id': 'req-1915657d-bc1f-4e61-804d-1cb381ddafa8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:09.509451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1915657d-bc1f-4e61-804d-1cb381ddafa8 admin admin] The token cache is being invalidate because project 72b3ae2eaa3c4a16898b763255335503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:25:09.510089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1915657d-bc1f-4e61-804d-1cb381ddafa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:09.511068 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1915657d-bc1f-4e61-804d-1cb381ddafa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:09.513409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1081/2166] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:25:09 2022] DELETE /identity/v3/projects/72b3ae2eaa3c4a16898b763255335503 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:25:09.521002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb656dc-36ab-47bf-b2f0-88a0e707ec7e None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:09.521696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb656dc-36ab-47bf-b2f0-88a0e707ec7e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:09.521696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb656dc-36ab-47bf-b2f0-88a0e707ec7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:09.547960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5bb656dc-36ab-47bf-b2f0-88a0e707ec7e None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:09.549542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5bb656dc-36ab-47bf-b2f0-88a0e707ec7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:09.638073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1086/2167] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:09.640787 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1842734-673c-458f-b6fa-d7400ddff7dc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:09.641127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1842734-673c-458f-b6fa-d7400ddff7dc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:09.641586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1842734-673c-458f-b6fa-d7400ddff7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:09.668098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d1842734-673c-458f-b6fa-d7400ddff7dc None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:09.670246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d1842734-673c-458f-b6fa-d7400ddff7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:09.685033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1082/2168] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:09.689878 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb656dc-36ab-47bf-b2f0-88a0e707ec7e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:09.704177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb656dc-36ab-47bf-b2f0-88a0e707ec7e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:09.710434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447f6ea-94d9-4fc6-bc16-2f977464e7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:09.711171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447f6ea-94d9-4fc6-bc16-2f977464e7ce admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:09.712373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447f6ea-94d9-4fc6-bc16-2f977464e7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:09.712373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447f6ea-94d9-4fc6-bc16-2f977464e7ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:09.712685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447f6ea-94d9-4fc6-bc16-2f977464e7ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:25:09.712954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447f6ea-94d9-4fc6-bc16-2f977464e7ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:09.714732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447f6ea-94d9-4fc6-bc16-2f977464e7ce admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:09.735637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1087/2169] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:25:09.743456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1842734-673c-458f-b6fa-d7400ddff7dc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:09.778491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1842734-673c-458f-b6fa-d7400ddff7dc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:09.793158 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20710b68-c81d-4b03-87de-726b309f8763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:09.795988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20710b68-c81d-4b03-87de-726b309f8763 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:09.796582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20710b68-c81d-4b03-87de-726b309f8763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:09.797679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20710b68-c81d-4b03-87de-726b309f8763 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:09.798791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20710b68-c81d-4b03-87de-726b309f8763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:09.809734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20710b68-c81d-4b03-87de-726b309f8763 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:09.884536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1083/2170] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:25:09 2022] POST /identity/v3/projects => generated 397 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:09.911551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447f6ea-94d9-4fc6-bc16-2f977464e7ce admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:09.953715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447f6ea-94d9-4fc6-bc16-2f977464e7ce admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:10.104492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1684ea-b016-45c7-9bb3-c7ea013106a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:10.107551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1684ea-b016-45c7-9bb3-c7ea013106a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:10.108731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1684ea-b016-45c7-9bb3-c7ea013106a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:10.108731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1684ea-b016-45c7-9bb3-c7ea013106a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:10.108731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1684ea-b016-45c7-9bb3-c7ea013106a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:10.112687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1684ea-b016-45c7-9bb3-c7ea013106a6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:10.184761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1088/2171] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:25:09 2022] POST /identity/v3/users => generated 299 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:10.191470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20710b68-c81d-4b03-87de-726b309f8763 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:10.213442 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20710b68-c81d-4b03-87de-726b309f8763 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:10.221516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41b95c-e395-4bd6-a294-bc1c927f50b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:10.223209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41b95c-e395-4bd6-a294-bc1c927f50b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:10.223660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41b95c-e395-4bd6-a294-bc1c927f50b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:10.223920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41b95c-e395-4bd6-a294-bc1c927f50b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:10.224762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41b95c-e395-4bd6-a294-bc1c927f50b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:10.226454 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41b95c-e395-4bd6-a294-bc1c927f50b5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:10.239702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1084/2172] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:25:10.245151 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1684ea-b016-45c7-9bb3-c7ea013106a6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:10.263282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1684ea-b016-45c7-9bb3-c7ea013106a6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:10.270543 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51525ba8-9d79-40b8-9753-9dca94290733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:10.271308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51525ba8-9d79-40b8-9753-9dca94290733 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:10.271635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51525ba8-9d79-40b8-9753-9dca94290733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:10.271907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51525ba8-9d79-40b8-9753-9dca94290733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:10.273045 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51525ba8-9d79-40b8-9753-9dca94290733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:10.274633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51525ba8-9d79-40b8-9753-9dca94290733 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:10.277117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:10.296631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1089/2173] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:25:10.302949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41b95c-e395-4bd6-a294-bc1c927f50b5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:10.325976 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41b95c-e395-4bd6-a294-bc1c927f50b5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:10.337002 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be2c61-6f59-4061-aa73-f7fcdee6d3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:10.338032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be2c61-6f59-4061-aa73-f7fcdee6d3da admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:10.338526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be2c61-6f59-4061-aa73-f7fcdee6d3da admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:10.338947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be2c61-6f59-4061-aa73-f7fcdee6d3da admin admin] PATH_INFO: `/v3/projects/104fca0150014dac9f38fac4f2f4a490/users/47e8b1f4d0454fb6a2886065bc9a9c23/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:10.374521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be2c61-6f59-4061-aa73-f7fcdee6d3da admin admin] RBAC: Authorizing `identity:create_grant(project_id=104fca0150014dac9f38fac4f2f4a490, user_id=47e8b1f4d0454fb6a2886065bc9a9c23, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:10.378095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be2c61-6f59-4061-aa73-f7fcdee6d3da admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:10.435806 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1085/2174] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:25:10 2022] PUT /identity/v3/projects/104fca0150014dac9f38fac4f2f4a490/users/47e8b1f4d0454fb6a2886065bc9a9c23/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:10.447056 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc28dc-f53d-4529-8035-b26d663ac576 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:10.447987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc28dc-f53d-4529-8035-b26d663ac576 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:10.448493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc28dc-f53d-4529-8035-b26d663ac576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:10.513484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-12bc28dc-f53d-4529-8035-b26d663ac576 None None] MFA Rules not processed for user `47e8b1f4d0454fb6a2886065bc9a9c23`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:10.515732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-12bc28dc-f53d-4529-8035-b26d663ac576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:10.626818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1090/2175] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:10 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:10.643590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be2c61-6f59-4061-aa73-f7fcdee6d3da admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:10.646018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f7be2c61-6f59-4061-aa73-f7fcdee6d3da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:10.727695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be2c61-6f59-4061-aa73-f7fcdee6d3da admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:10.732294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d542adb-1bdd-4e04-bf35-2cd96b37fb5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:10.732927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d542adb-1bdd-4e04-bf35-2cd96b37fb5f service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:10.733227 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d542adb-1bdd-4e04-bf35-2cd96b37fb5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:10.733498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d542adb-1bdd-4e04-bf35-2cd96b37fb5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:10.746437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d542adb-1bdd-4e04-bf35-2cd96b37fb5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:10.748662 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d542adb-1bdd-4e04-bf35-2cd96b37fb5f service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:10.761020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1086/2176] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:25:10 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:25:19.474459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc28dc-f53d-4529-8035-b26d663ac576 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:19.585454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc28dc-f53d-4529-8035-b26d663ac576 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:19.592837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:19.594104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:19.595123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:19.595123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] PATH_INFO: `/v3/users/47e8b1f4d0454fb6a2886065bc9a9c23` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:19.597087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47e8b1f4d0454fb6a2886065bc9a9c23)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:19.599302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:19.688195 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47e8b1f4d0454fb6a2886065bc9a9c23'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:19.702382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47e8b1f4d0454fb6a2886065bc9a9c23'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:19.703723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47e8b1f4d0454fb6a2886065bc9a9c23'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:19.760001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47e8b1f4d0454fb6a2886065bc9a9c23'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:19.791871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47e8b1f4d0454fb6a2886065bc9a9c23'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:19.810226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1091/2177] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:25:19 2022] DELETE /identity/v3/users/47e8b1f4d0454fb6a2886065bc9a9c23 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:19.820467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f6ba5-f454-42c2-abd3-ddfb35bb1d2a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:19.821008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f6ba5-f454-42c2-abd3-ddfb35bb1d2a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:19.821456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f6ba5-f454-42c2-abd3-ddfb35bb1d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:19.876956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c86f6ba5-f454-42c2-abd3-ddfb35bb1d2a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:19.881928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c86f6ba5-f454-42c2-abd3-ddfb35bb1d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:19.926524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:19.988104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1087/2178] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:20.004958 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:20.010775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:20.027382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:20.122361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930bba9-c507-40e8-9cbe-47ca44a68166 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:20.131280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99e0ab-fe22-4c62-96c4-46de46d15bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:20.131996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99e0ab-fe22-4c62-96c4-46de46d15bca service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:20.132340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99e0ab-fe22-4c62-96c4-46de46d15bca service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:20.132689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99e0ab-fe22-4c62-96c4-46de46d15bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:20.144048 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99e0ab-fe22-4c62-96c4-46de46d15bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:20.145910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99e0ab-fe22-4c62-96c4-46de46d15bca service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:20.163042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1092/2179] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:25:20.600442 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f6ba5-f454-42c2-abd3-ddfb35bb1d2a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:20.601934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c86f6ba5-f454-42c2-abd3-ddfb35bb1d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:20.614779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f6ba5-f454-42c2-abd3-ddfb35bb1d2a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:20.620245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729eeac-7d20-4933-9525-299c1cc9804c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:20.621001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729eeac-7d20-4933-9525-299c1cc9804c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:20.621890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729eeac-7d20-4933-9525-299c1cc9804c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:20.621890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729eeac-7d20-4933-9525-299c1cc9804c admin admin] PATH_INFO: `/v3/projects/104fca0150014dac9f38fac4f2f4a490` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:20.623059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729eeac-7d20-4933-9525-299c1cc9804c admin admin] RBAC: Authorizing `identity:delete_project(project_id=104fca0150014dac9f38fac4f2f4a490)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:20.625101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729eeac-7d20-4933-9525-299c1cc9804c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:20.691603 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3729eeac-7d20-4933-9525-299c1cc9804c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '104fca0150014dac9f38fac4f2f4a490', 'request_id': 'req-3729eeac-7d20-4933-9525-299c1cc9804c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:20.700813 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3729eeac-7d20-4933-9525-299c1cc9804c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '104fca0150014dac9f38fac4f2f4a490', 'request_id': 'req-3729eeac-7d20-4933-9525-299c1cc9804c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:20.707495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3729eeac-7d20-4933-9525-299c1cc9804c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '104fca0150014dac9f38fac4f2f4a490', 'request_id': 'req-3729eeac-7d20-4933-9525-299c1cc9804c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:20.724115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3729eeac-7d20-4933-9525-299c1cc9804c admin admin] The token cache is being invalidate because project 104fca0150014dac9f38fac4f2f4a490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:25:20.724402 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3729eeac-7d20-4933-9525-299c1cc9804c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:20.725062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3729eeac-7d20-4933-9525-299c1cc9804c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:20.726601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1088/2180] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:25:20 2022] DELETE /identity/v3/projects/104fca0150014dac9f38fac4f2f4a490 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:25:20.731591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc124b29-8fbc-4929-87fd-39c1e204f820 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:20.732132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc124b29-8fbc-4929-87fd-39c1e204f820 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:20.732132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc124b29-8fbc-4929-87fd-39c1e204f820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:20.757716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-dc124b29-8fbc-4929-87fd-39c1e204f820 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:20.759649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-dc124b29-8fbc-4929-87fd-39c1e204f820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:20.867126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1093/2181] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:20.876865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf39bcc-d57f-4d30-a64a-c3ec6c1ae96f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:20.877469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf39bcc-d57f-4d30-a64a-c3ec6c1ae96f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:20.877469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf39bcc-d57f-4d30-a64a-c3ec6c1ae96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:20.915658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cdf39bcc-d57f-4d30-a64a-c3ec6c1ae96f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:20.917162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cdf39bcc-d57f-4d30-a64a-c3ec6c1ae96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:20.927049 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1089/2182] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:20.932528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc124b29-8fbc-4929-87fd-39c1e204f820 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:20.954001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc124b29-8fbc-4929-87fd-39c1e204f820 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:20.963980 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a3b44-ced4-49df-b2ea-bd5d30c6e36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:20.964985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a3b44-ced4-49df-b2ea-bd5d30c6e36a admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:20.965429 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a3b44-ced4-49df-b2ea-bd5d30c6e36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:20.965898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a3b44-ced4-49df-b2ea-bd5d30c6e36a admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:20.967062 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a3b44-ced4-49df-b2ea-bd5d30c6e36a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:25:20.967583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a3b44-ced4-49df-b2ea-bd5d30c6e36a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:20.970328 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a3b44-ced4-49df-b2ea-bd5d30c6e36a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:20.994210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1094/2183] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:25:20.995927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf39bcc-d57f-4d30-a64a-c3ec6c1ae96f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:21.010019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf39bcc-d57f-4d30-a64a-c3ec6c1ae96f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:21.018541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957f80a-f300-4ecd-8ee5-55aec1eddb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:21.019677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957f80a-f300-4ecd-8ee5-55aec1eddb73 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:21.023852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957f80a-f300-4ecd-8ee5-55aec1eddb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:21.024215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957f80a-f300-4ecd-8ee5-55aec1eddb73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:21.025007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957f80a-f300-4ecd-8ee5-55aec1eddb73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:21.027063 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957f80a-f300-4ecd-8ee5-55aec1eddb73 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:21.064312 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1090/2184] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:25:20 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:21.069374 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a3b44-ced4-49df-b2ea-bd5d30c6e36a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:21.081227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a3b44-ced4-49df-b2ea-bd5d30c6e36a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:21.147609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ccd47-df0f-4681-9cd6-937ff1c8831d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:21.148263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ccd47-df0f-4681-9cd6-937ff1c8831d admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:21.148541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ccd47-df0f-4681-9cd6-937ff1c8831d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:21.148813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ccd47-df0f-4681-9cd6-937ff1c8831d admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:21.149448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ccd47-df0f-4681-9cd6-937ff1c8831d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:21.151457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ccd47-df0f-4681-9cd6-937ff1c8831d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:21.178781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1095/2185] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:25:21 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:21.183034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957f80a-f300-4ecd-8ee5-55aec1eddb73 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:21.193793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957f80a-f300-4ecd-8ee5-55aec1eddb73 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:21.201095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ebc45-84a4-4452-8625-434841674eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:21.201726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ebc45-84a4-4452-8625-434841674eef admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:21.202191 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ebc45-84a4-4452-8625-434841674eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:21.202468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ebc45-84a4-4452-8625-434841674eef admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:21.203389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ebc45-84a4-4452-8625-434841674eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:21.205025 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ebc45-84a4-4452-8625-434841674eef admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:21.219599 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1091/2186] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:25:21.224368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ccd47-df0f-4681-9cd6-937ff1c8831d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:21.236643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ccd47-df0f-4681-9cd6-937ff1c8831d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:21.242486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72d65a-152e-43ba-9e46-61847c3db948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:21.243287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72d65a-152e-43ba-9e46-61847c3db948 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:21.243503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72d65a-152e-43ba-9e46-61847c3db948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:21.243785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72d65a-152e-43ba-9e46-61847c3db948 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:21.244570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72d65a-152e-43ba-9e46-61847c3db948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:21.246284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72d65a-152e-43ba-9e46-61847c3db948 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:21.265045 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1096/2187] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:25:21.267700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ebc45-84a4-4452-8625-434841674eef admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:21.282792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ebc45-84a4-4452-8625-434841674eef admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:21.289807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bdfcf-c2e3-4ca7-9fcb-1385bcfad226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:21.290665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bdfcf-c2e3-4ca7-9fcb-1385bcfad226 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:21.291023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bdfcf-c2e3-4ca7-9fcb-1385bcfad226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:21.291376 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bdfcf-c2e3-4ca7-9fcb-1385bcfad226 admin admin] PATH_INFO: `/v3/projects/f1939a4a2ffd40c49d04b42983fba762/users/70d318fe37c344649a165ef9a06a7b74/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:21.311477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bdfcf-c2e3-4ca7-9fcb-1385bcfad226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1939a4a2ffd40c49d04b42983fba762, user_id=70d318fe37c344649a165ef9a06a7b74, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:21.314262 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bdfcf-c2e3-4ca7-9fcb-1385bcfad226 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:21.337856 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1092/2188] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:25:21 2022] PUT /identity/v3/projects/f1939a4a2ffd40c49d04b42983fba762/users/70d318fe37c344649a165ef9a06a7b74/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:21.345732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54c765-d3f6-40ae-823d-7019fceee9bd None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:21.346026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54c765-d3f6-40ae-823d-7019fceee9bd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:21.346392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54c765-d3f6-40ae-823d-7019fceee9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:21.370903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-de54c765-d3f6-40ae-823d-7019fceee9bd None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:21.372163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-de54c765-d3f6-40ae-823d-7019fceee9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:21.384719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:21.493037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1097/2189] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:21.503611 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bdfcf-c2e3-4ca7-9fcb-1385bcfad226 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:21.505668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ee5bdfcf-c2e3-4ca7-9fcb-1385bcfad226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:21.597774 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:21.608120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bdfcf-c2e3-4ca7-9fcb-1385bcfad226 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:21.617583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6552-c4ab-427f-9728-f0bae7cd8ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:21.618605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6552-c4ab-427f-9728-f0bae7cd8ceb service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:21.619151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6552-c4ab-427f-9728-f0bae7cd8ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:21.619587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6552-c4ab-427f-9728-f0bae7cd8ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:21.636521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb6552-c4ab-427f-9728-f0bae7cd8ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:21.640075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb6552-c4ab-427f-9728-f0bae7cd8ceb service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:21.664418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1093/2190] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:25:29.279628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7eee8f-d89b-4d9f-bf61-6a61accc4f4b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:29.280459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7eee8f-d89b-4d9f-bf61-6a61accc4f4b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:29.280698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7eee8f-d89b-4d9f-bf61-6a61accc4f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:29.320984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9e7eee8f-d89b-4d9f-bf61-6a61accc4f4b None None] MFA Rules not processed for user `70d318fe37c344649a165ef9a06a7b74`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:29.322295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9e7eee8f-d89b-4d9f-bf61-6a61accc4f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:29.415367 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1098/2191] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:29.417381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6552-c4ab-427f-9728-f0bae7cd8ceb service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:29.429069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6552-c4ab-427f-9728-f0bae7cd8ceb service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:29.434103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c0c24-10b5-4100-a553-54c210299995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:29.434736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c0c24-10b5-4100-a553-54c210299995 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:29.435035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c0c24-10b5-4100-a553-54c210299995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:29.436832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c0c24-10b5-4100-a553-54c210299995 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:29.436832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c0c24-10b5-4100-a553-54c210299995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:29.437893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c0c24-10b5-4100-a553-54c210299995 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:29.471244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1094/2192] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:25:29 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:29.475097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7eee8f-d89b-4d9f-bf61-6a61accc4f4b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:29.492123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7eee8f-d89b-4d9f-bf61-6a61accc4f4b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:29.594948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a928c9-d39d-449c-ae9b-5e230b6a9bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:29.595277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a928c9-d39d-449c-ae9b-5e230b6a9bfe admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:29.596299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a928c9-d39d-449c-ae9b-5e230b6a9bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:29.596750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a928c9-d39d-449c-ae9b-5e230b6a9bfe admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:29.597424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a928c9-d39d-449c-ae9b-5e230b6a9bfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:29.599438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a928c9-d39d-449c-ae9b-5e230b6a9bfe admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:29.634347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1099/2193] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:25:29 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:25:29.640715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c0c24-10b5-4100-a553-54c210299995 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:29.651481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c0c24-10b5-4100-a553-54c210299995 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:29.657129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13ef97-3674-4a0b-8904-8cf40be91d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:29.657749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13ef97-3674-4a0b-8904-8cf40be91d4f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:29.658572 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13ef97-3674-4a0b-8904-8cf40be91d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:29.658845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13ef97-3674-4a0b-8904-8cf40be91d4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:29.659647 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13ef97-3674-4a0b-8904-8cf40be91d4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:29.661272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13ef97-3674-4a0b-8904-8cf40be91d4f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:29.677810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1095/2194] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:25:29.680086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a928c9-d39d-449c-ae9b-5e230b6a9bfe admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:29.699111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a928c9-d39d-449c-ae9b-5e230b6a9bfe admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:29.704549 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84192d-bb61-426c-b156-d7ee7e07322a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:29.705230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84192d-bb61-426c-b156-d7ee7e07322a admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:29.705543 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84192d-bb61-426c-b156-d7ee7e07322a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:29.705771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84192d-bb61-426c-b156-d7ee7e07322a admin admin] PATH_INFO: `/v3/projects/36e39fe8818a4b5697552d963c19f53b/users/3eea0f656490491e8fc151ebbb47769f/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:29.742377 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84192d-bb61-426c-b156-d7ee7e07322a admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e39fe8818a4b5697552d963c19f53b, user_id=3eea0f656490491e8fc151ebbb47769f, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:29.747433 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84192d-bb61-426c-b156-d7ee7e07322a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:29.781777 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1100/2195] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:25:29 2022] PUT /identity/v3/projects/36e39fe8818a4b5697552d963c19f53b/users/3eea0f656490491e8fc151ebbb47769f/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:29.788063 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13ef97-3674-4a0b-8904-8cf40be91d4f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:29.800040 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13ef97-3674-4a0b-8904-8cf40be91d4f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:29.878625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e553e9-950d-4173-bd49-eedfe05cf13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:29.879348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e553e9-950d-4173-bd49-eedfe05cf13f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:29.879625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e553e9-950d-4173-bd49-eedfe05cf13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:29.879901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e553e9-950d-4173-bd49-eedfe05cf13f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:29.880555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e553e9-950d-4173-bd49-eedfe05cf13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:29.882253 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e553e9-950d-4173-bd49-eedfe05cf13f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:29.899621 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1096/2196] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:25:29.901211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84192d-bb61-426c-b156-d7ee7e07322a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:29.920114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84192d-bb61-426c-b156-d7ee7e07322a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:29.927057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee97d2-18be-439d-bbd1-55c3d8a5f174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:29.928142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee97d2-18be-439d-bbd1-55c3d8a5f174 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:29.928664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee97d2-18be-439d-bbd1-55c3d8a5f174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:29.929051 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee97d2-18be-439d-bbd1-55c3d8a5f174 admin admin] PATH_INFO: `/v3/projects/36e39fe8818a4b5697552d963c19f53b/users/3eea0f656490491e8fc151ebbb47769f/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:29.931524 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee97d2-18be-439d-bbd1-55c3d8a5f174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e39fe8818a4b5697552d963c19f53b, user_id=3eea0f656490491e8fc151ebbb47769f, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:29.934654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee97d2-18be-439d-bbd1-55c3d8a5f174 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:29.942231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:29.951518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1101/2197] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:25:29 2022] PUT /identity/v3/projects/36e39fe8818a4b5697552d963c19f53b/users/3eea0f656490491e8fc151ebbb47769f/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:36.522342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e553e9-950d-4173-bd49-eedfe05cf13f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:36.522342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f8e553e9-950d-4173-bd49-eedfe05cf13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:36.539907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e553e9-950d-4173-bd49-eedfe05cf13f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:36.708021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:36.709156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:36.709650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:36.710335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] PATH_INFO: `/v3/users/35d1637fd36d42e78c733ce49a2ce845` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:36.712505 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35d1637fd36d42e78c733ce49a2ce845)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:36.716393 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:36.824161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35d1637fd36d42e78c733ce49a2ce845'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:36.839924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35d1637fd36d42e78c733ce49a2ce845'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:36.843439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:36.872552 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35d1637fd36d42e78c733ce49a2ce845'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:36.873948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35d1637fd36d42e78c733ce49a2ce845'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:36.896220 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35d1637fd36d42e78c733ce49a2ce845'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:36.924615 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1097/2198] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:25:36 2022] DELETE /identity/v3/users/35d1637fd36d42e78c733ce49a2ce845 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:36.942111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18901f-595f-45db-bd91-c59aee4ca2fb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:36.942944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18901f-595f-45db-bd91-c59aee4ca2fb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:36.943259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18901f-595f-45db-bd91-c59aee4ca2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:37.085263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6c18901f-595f-45db-bd91-c59aee4ca2fb None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:37.087796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6c18901f-595f-45db-bd91-c59aee4ca2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:37.291438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1102/2199] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:37.312738 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:37.321408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:37.496545 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ce911-9f44-404a-b6cb-a085e01a11a9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:37.530282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59538ca-2d4a-4e54-8652-aa9bf1f7075a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:37.537126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59538ca-2d4a-4e54-8652-aa9bf1f7075a service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:37.539681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59538ca-2d4a-4e54-8652-aa9bf1f7075a service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:37.540658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59538ca-2d4a-4e54-8652-aa9bf1f7075a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:37.592509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59538ca-2d4a-4e54-8652-aa9bf1f7075a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:37.595498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59538ca-2d4a-4e54-8652-aa9bf1f7075a service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:37.617910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1098/2200] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:25:38.407444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18901f-595f-45db-bd91-c59aee4ca2fb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:38.412854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6c18901f-595f-45db-bd91-c59aee4ca2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:38.453822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18901f-595f-45db-bd91-c59aee4ca2fb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:38.471078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d741a-55e2-4500-bbb2-f60e717a9e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:38.472154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d741a-55e2-4500-bbb2-f60e717a9e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:38.472616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d741a-55e2-4500-bbb2-f60e717a9e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:38.472886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d741a-55e2-4500-bbb2-f60e717a9e2c admin admin] PATH_INFO: `/v3/projects/41ad661551c2451b87557b0b97491df6` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:38.474414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d741a-55e2-4500-bbb2-f60e717a9e2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ad661551c2451b87557b0b97491df6)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:38.477115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d741a-55e2-4500-bbb2-f60e717a9e2c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:38.621047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-569d741a-55e2-4500-bbb2-f60e717a9e2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ad661551c2451b87557b0b97491df6', 'request_id': 'req-569d741a-55e2-4500-bbb2-f60e717a9e2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:38.640452 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-569d741a-55e2-4500-bbb2-f60e717a9e2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ad661551c2451b87557b0b97491df6', 'request_id': 'req-569d741a-55e2-4500-bbb2-f60e717a9e2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:38.658405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-569d741a-55e2-4500-bbb2-f60e717a9e2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ad661551c2451b87557b0b97491df6', 'request_id': 'req-569d741a-55e2-4500-bbb2-f60e717a9e2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:38.693693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-569d741a-55e2-4500-bbb2-f60e717a9e2c admin admin] The token cache is being invalidate because project 41ad661551c2451b87557b0b97491df6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:25:38.695918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-569d741a-55e2-4500-bbb2-f60e717a9e2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:38.697238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-569d741a-55e2-4500-bbb2-f60e717a9e2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:38.700145 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1103/2201] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:25:38 2022] DELETE /identity/v3/projects/41ad661551c2451b87557b0b97491df6 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:25:38.706460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34833c-aeab-40d3-a65c-6719a468e212 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:38.707324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34833c-aeab-40d3-a65c-6719a468e212 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:38.707760 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d34833c-aeab-40d3-a65c-6719a468e212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:38.740486 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3d34833c-aeab-40d3-a65c-6719a468e212 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:38.742274 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3d34833c-aeab-40d3-a65c-6719a468e212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:38.915427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1099/2202] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:38.924930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01593c-a2ce-4d39-b470-cdf47ff1f46e None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:38.925327 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01593c-a2ce-4d39-b470-cdf47ff1f46e None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:38.925674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01593c-a2ce-4d39-b470-cdf47ff1f46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:38.998426 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6d01593c-a2ce-4d39-b470-cdf47ff1f46e None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:39.001822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6d01593c-a2ce-4d39-b470-cdf47ff1f46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:39.024365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1104/2203] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:39.035688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34833c-aeab-40d3-a65c-6719a468e212 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:39.053897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d34833c-aeab-40d3-a65c-6719a468e212 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:39.072037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212862d3-f75b-488d-912e-7fa8aa771a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:39.072682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212862d3-f75b-488d-912e-7fa8aa771a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:39.072971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212862d3-f75b-488d-912e-7fa8aa771a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:39.073350 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212862d3-f75b-488d-912e-7fa8aa771a29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:39.074155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212862d3-f75b-488d-912e-7fa8aa771a29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:25:39.074587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212862d3-f75b-488d-912e-7fa8aa771a29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:39.076481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212862d3-f75b-488d-912e-7fa8aa771a29 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:39.175108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01593c-a2ce-4d39-b470-cdf47ff1f46e None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:39.187408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1100/2204] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:25:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:25:39.198210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01593c-a2ce-4d39-b470-cdf47ff1f46e None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:39.223094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480f7e8-a653-4965-a2db-7d671ed843fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:39.226571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480f7e8-a653-4965-a2db-7d671ed843fa admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:39.229445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480f7e8-a653-4965-a2db-7d671ed843fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:39.229697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480f7e8-a653-4965-a2db-7d671ed843fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:39.232552 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480f7e8-a653-4965-a2db-7d671ed843fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:39.236044 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480f7e8-a653-4965-a2db-7d671ed843fa admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:39.255320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c09538-90c8-453e-ad6b-293251153c5c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:39.256240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c09538-90c8-453e-ad6b-293251153c5c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:39.256648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c09538-90c8-453e-ad6b-293251153c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:39.300536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1105/2205] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:25:39 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:39.311300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-15c09538-90c8-453e-ad6b-293251153c5c None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:39.311619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480f7e8-a653-4965-a2db-7d671ed843fa admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:39.314469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-15c09538-90c8-453e-ad6b-293251153c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:39.395424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:39.416876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480f7e8-a653-4965-a2db-7d671ed843fa admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:39.433639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65957dd-c4cf-468f-9331-f92787f3bf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:39.436125 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65957dd-c4cf-468f-9331-f92787f3bf64 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:39.439430 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65957dd-c4cf-468f-9331-f92787f3bf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:39.440049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65957dd-c4cf-468f-9331-f92787f3bf64 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:39.440993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65957dd-c4cf-468f-9331-f92787f3bf64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:39.444120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65957dd-c4cf-468f-9331-f92787f3bf64 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:39.480728 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1106/2206] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:25:39 2022] POST /identity/v3/users => generated 288 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:25:39.485228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65957dd-c4cf-468f-9331-f92787f3bf64 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:39.513655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65957dd-c4cf-468f-9331-f92787f3bf64 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:39.521447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1101/2207] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:25:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:25:39.554565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a78bd-65be-4e30-8c2b-cb212baa7fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:39.556153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a78bd-65be-4e30-8c2b-cb212baa7fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:39.556539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a78bd-65be-4e30-8c2b-cb212baa7fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:39.556925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a78bd-65be-4e30-8c2b-cb212baa7fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:39.558400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a78bd-65be-4e30-8c2b-cb212baa7fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:39.560730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a78bd-65be-4e30-8c2b-cb212baa7fd6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:39.593746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c09538-90c8-453e-ad6b-293251153c5c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:39.596003 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-15c09538-90c8-453e-ad6b-293251153c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:39.609952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1107/2208] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:25:39.611680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a78bd-65be-4e30-8c2b-cb212baa7fd6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:39.611889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:39.628351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a78bd-65be-4e30-8c2b-cb212baa7fd6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:39.648295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b1a84-2e03-46d3-b958-5c71eb567505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:39.650563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b1a84-2e03-46d3-b958-5c71eb567505 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:39.650563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b1a84-2e03-46d3-b958-5c71eb567505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:39.650940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3b1a84-2e03-46d3-b958-5c71eb567505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:39.651343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b1a84-2e03-46d3-b958-5c71eb567505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:39.655488 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3b1a84-2e03-46d3-b958-5c71eb567505 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:39.681446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1108/2209] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:25:39.682286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b1a84-2e03-46d3-b958-5c71eb567505 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:39.703844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3b1a84-2e03-46d3-b958-5c71eb567505 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:39.719119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183328a-21ff-45cd-873c-05cb7baaefff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:39.719119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183328a-21ff-45cd-873c-05cb7baaefff admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:39.719119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183328a-21ff-45cd-873c-05cb7baaefff admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:39.719119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183328a-21ff-45cd-873c-05cb7baaefff admin admin] PATH_INFO: `/v3/projects/2a74ec2f4aa449ae8ff41b3d1b9d7c9b/users/30c4ca3c226a4910a08b881022039277/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:39.740444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c09538-90c8-453e-ad6b-293251153c5c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:39.743349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183328a-21ff-45cd-873c-05cb7baaefff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a74ec2f4aa449ae8ff41b3d1b9d7c9b, user_id=30c4ca3c226a4910a08b881022039277, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:39.748985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3b21e-fbea-445d-b7cf-5a72425dafd0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:39.749736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3b21e-fbea-445d-b7cf-5a72425dafd0 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:39.750088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3b21e-fbea-445d-b7cf-5a72425dafd0 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:39.750572 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3b21e-fbea-445d-b7cf-5a72425dafd0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:39.754888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183328a-21ff-45cd-873c-05cb7baaefff admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:39.762858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3b21e-fbea-445d-b7cf-5a72425dafd0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:39.765082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3b21e-fbea-445d-b7cf-5a72425dafd0 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:39.788506 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1102/2211] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:25:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 16:25:39.790379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1109/2211] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:25:39 2022] PUT /identity/v3/projects/2a74ec2f4aa449ae8ff41b3d1b9d7c9b/users/30c4ca3c226a4910a08b881022039277/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:39.803068 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc417fef-2940-4a91-923b-6750ce6cdc1e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:39.803946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc417fef-2940-4a91-923b-6750ce6cdc1e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:39.804299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc417fef-2940-4a91-923b-6750ce6cdc1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:39.877994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cc417fef-2940-4a91-923b-6750ce6cdc1e None None] MFA Rules not processed for user `30c4ca3c226a4910a08b881022039277`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:39.877994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cc417fef-2940-4a91-923b-6750ce6cdc1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:40.052093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1103/2212] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:40.061136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183328a-21ff-45cd-873c-05cb7baaefff admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:40.065032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6183328a-21ff-45cd-873c-05cb7baaefff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:40.335856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183328a-21ff-45cd-873c-05cb7baaefff admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:40.350967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef8f55-640c-4834-96a5-7d3e46e37d78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:40.352040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef8f55-640c-4834-96a5-7d3e46e37d78 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:40.352697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef8f55-640c-4834-96a5-7d3e46e37d78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:40.353192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef8f55-640c-4834-96a5-7d3e46e37d78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:40.384571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef8f55-640c-4834-96a5-7d3e46e37d78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:40.388854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef8f55-640c-4834-96a5-7d3e46e37d78 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:40.446969 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1110/2213] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:25:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:25:41.113768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666a4ae-40d5-49ed-84ff-af9f6fbf2f1a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:41.114998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666a4ae-40d5-49ed-84ff-af9f6fbf2f1a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:41.115431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666a4ae-40d5-49ed-84ff-af9f6fbf2f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:41.245139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0666a4ae-40d5-49ed-84ff-af9f6fbf2f1a None None] MFA Rules not processed for user `3eea0f656490491e8fc151ebbb47769f`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:41.252494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0666a4ae-40d5-49ed-84ff-af9f6fbf2f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:41.645196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1104/2214] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 533 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:25:41.687114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef8f55-640c-4834-96a5-7d3e46e37d78 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:41.688627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-28ef8f55-640c-4834-96a5-7d3e46e37d78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:41.948534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef8f55-640c-4834-96a5-7d3e46e37d78 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:41.962654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488f87a-d892-445d-bb63-997592e369ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2d76fb9cbdd40f9b2a748209cee5b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:41.963878 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488f87a-d892-445d-bb63-997592e369ac service glance] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:41.964355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488f87a-d892-445d-bb63-997592e369ac service glance] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:41.964836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488f87a-d892-445d-bb63-997592e369ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:41.978032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1488f87a-d892-445d-bb63-997592e369ac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:42.172783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:42.184310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488f87a-d892-445d-bb63-997592e369ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:42.186698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1488f87a-d892-445d-bb63-997592e369ac service glance] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:42.218800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1111/2215] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:25:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:25:42.716631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb255e5-3d7f-4812-92e4-de7286e8fd7c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:42.717645 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb255e5-3d7f-4812-92e4-de7286e8fd7c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:42.718151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb255e5-3d7f-4812-92e4-de7286e8fd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:42.759896 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2fb255e5-3d7f-4812-92e4-de7286e8fd7c None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:42.762207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2fb255e5-3d7f-4812-92e4-de7286e8fd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:42.769904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:42.981383 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1105/2216] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:25:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:25:43.068841 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488f87a-d892-445d-bb63-997592e369ac service glance] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:43.328001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1488f87a-d892-445d-bb63-997592e369ac service glance] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:43.337276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532eb9e-28b8-4105-b61c-2661be31f631 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:43.338504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532eb9e-28b8-4105-b61c-2661be31f631 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:43.338965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532eb9e-28b8-4105-b61c-2661be31f631 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:43.339604 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532eb9e-28b8-4105-b61c-2661be31f631 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:43.370714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532eb9e-28b8-4105-b61c-2661be31f631 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:43.372597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532eb9e-28b8-4105-b61c-2661be31f631 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:43.395039 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1112/2217] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:25:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 16:25:52.241643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d939d05-204a-44f6-8ea2-a3a0da90256b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:52.242320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d939d05-204a-44f6-8ea2-a3a0da90256b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:52.243183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d939d05-204a-44f6-8ea2-a3a0da90256b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:52.341436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7d939d05-204a-44f6-8ea2-a3a0da90256b None None] MFA Rules not processed for user `b917d84a650f4355a676e315ea039f0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:52.342796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7d939d05-204a-44f6-8ea2-a3a0da90256b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:52.439549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1106/2218] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:53.386078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=35 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:57.673201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b314b-d498-47db-9898-bf0a9fe280dd None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:57.674835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b314b-d498-47db-9898-bf0a9fe280dd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:57.675566 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b314b-d498-47db-9898-bf0a9fe280dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:57.780793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-769b314b-d498-47db-9898-bf0a9fe280dd None None] MFA Rules not processed for user `d2382a6610ae42fa865283ed12dd9982`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:57.780793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-769b314b-d498-47db-9898-bf0a9fe280dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:57.932695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1113/2219] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:58.949248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d939d05-204a-44f6-8ea2-a3a0da90256b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:59.118642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d939d05-204a-44f6-8ea2-a3a0da90256b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:59.133325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:59.133979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:59.136444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:59.137577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] PATH_INFO: `/v3/users/30c4ca3c226a4910a08b881022039277` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:59.139135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c4ca3c226a4910a08b881022039277)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:59.168480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:59.299572 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c4ca3c226a4910a08b881022039277'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:59.315967 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c4ca3c226a4910a08b881022039277'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:59.332099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:25:59.350525 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c4ca3c226a4910a08b881022039277'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:59.355683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c4ca3c226a4910a08b881022039277'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:59.374124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c4ca3c226a4910a08b881022039277'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:25:59.384084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1107/2220] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:25:58 2022] DELETE /identity/v3/users/30c4ca3c226a4910a08b881022039277 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:25:59.388317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdce626-58f7-41dd-a80e-79922d088a0d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:59.388913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdce626-58f7-41dd-a80e-79922d088a0d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:59.389115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdce626-58f7-41dd-a80e-79922d088a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:59.422787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9fdce626-58f7-41dd-a80e-79922d088a0d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:25:59.424335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9fdce626-58f7-41dd-a80e-79922d088a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:59.555833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1114/2221] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:25:59.568754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:25:59.573416 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:25:59.648167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1332e0-bcc9-4e07-ab06-75897b672465 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:25:59.651919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228118bf-f72d-4a25-a17b-6372f4f3e275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:25:59.652539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228118bf-f72d-4a25-a17b-6372f4f3e275 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:25:59.652816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228118bf-f72d-4a25-a17b-6372f4f3e275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:25:59.653087 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228118bf-f72d-4a25-a17b-6372f4f3e275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:25:59.664897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228118bf-f72d-4a25-a17b-6372f4f3e275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:25:59.670671 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228118bf-f72d-4a25-a17b-6372f4f3e275 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:25:59.705500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1108/2222] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:25:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:26:00.714369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdce626-58f7-41dd-a80e-79922d088a0d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:00.715743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9fdce626-58f7-41dd-a80e-79922d088a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:00.768253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdce626-58f7-41dd-a80e-79922d088a0d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:00.781342 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:00.782868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:00.783505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:00.784365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] PATH_INFO: `/v3/projects/2a74ec2f4aa449ae8ff41b3d1b9d7c9b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:00.786136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a74ec2f4aa449ae8ff41b3d1b9d7c9b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:00.788768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:00.928020 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a74ec2f4aa449ae8ff41b3d1b9d7c9b', 'request_id': 'req-c018fd82-db9d-4f3f-9326-6251c0294ed4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:00.938107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a74ec2f4aa449ae8ff41b3d1b9d7c9b', 'request_id': 'req-c018fd82-db9d-4f3f-9326-6251c0294ed4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:00.947554 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a74ec2f4aa449ae8ff41b3d1b9d7c9b', 'request_id': 'req-c018fd82-db9d-4f3f-9326-6251c0294ed4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:00.996676 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] The token cache is being invalidate because project 2a74ec2f4aa449ae8ff41b3d1b9d7c9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:26:00.997100 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:00.997851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:01.000473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1115/2223] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:26:00 2022] DELETE /identity/v3/projects/2a74ec2f4aa449ae8ff41b3d1b9d7c9b => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:26:02.084260 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369fc35-329b-4d4c-a4ba-8f145fdc6846 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:02.084733 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369fc35-329b-4d4c-a4ba-8f145fdc6846 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:02.085015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369fc35-329b-4d4c-a4ba-8f145fdc6846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:02.109719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c369fc35-329b-4d4c-a4ba-8f145fdc6846 None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:02.112018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c369fc35-329b-4d4c-a4ba-8f145fdc6846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:02.190572 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1109/2224] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:26:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:02.205281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:02.207025 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:02.251346 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:02.304005 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018fd82-db9d-4f3f-9326-6251c0294ed4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:02.309260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90262de8-c586-40bd-b28f-7297536633c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:02.310097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90262de8-c586-40bd-b28f-7297536633c3 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:02.310381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90262de8-c586-40bd-b28f-7297536633c3 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:02.310660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90262de8-c586-40bd-b28f-7297536633c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:02.320719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90262de8-c586-40bd-b28f-7297536633c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:02.322901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90262de8-c586-40bd-b28f-7297536633c3 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:02.331605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1116/2225] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:26:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 16:26:03.079862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ee333-a6f3-463d-bc0a-19137694765b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:03.080462 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ee333-a6f3-463d-bc0a-19137694765b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:03.081055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538ee333-a6f3-463d-bc0a-19137694765b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:03.158674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37f9a0-4514-4508-8fff-2a3c3f10ca31 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:03.158975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37f9a0-4514-4508-8fff-2a3c3f10ca31 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:03.159241 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37f9a0-4514-4508-8fff-2a3c3f10ca31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:03.168473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-538ee333-a6f3-463d-bc0a-19137694765b None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:03.174166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-538ee333-a6f3-463d-bc0a-19137694765b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:03.199683 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-1c37f9a0-4514-4508-8fff-2a3c3f10ca31 None None] MFA Rules not processed for user `70d318fe37c344649a165ef9a06a7b74`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:03.205970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1110/2226] 10.222.0.64 () {58 vars in 967 bytes} [Wed Sep 21 16:26:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:03.209578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1c37f9a0-4514-4508-8fff-2a3c3f10ca31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:03.229052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ee333-a6f3-463d-bc0a-19137694765b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:03.267101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538ee333-a6f3-463d-bc0a-19137694765b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:03.279350 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a6e57-4487-40cc-86f2-655080d0e314 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:03.281319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a6e57-4487-40cc-86f2-655080d0e314 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:03.282143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a6e57-4487-40cc-86f2-655080d0e314 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:03.282882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a6e57-4487-40cc-86f2-655080d0e314 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:03.301166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a6e57-4487-40cc-86f2-655080d0e314 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:03.304672 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a6e57-4487-40cc-86f2-655080d0e314 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:03.321564 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1111/2227] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:26:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 16:26:03.346153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1117/2228] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:26:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:26:03.660264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a4508-93ac-4aec-bb68-eec4d1e9940d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:03.660582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a4508-93ac-4aec-bb68-eec4d1e9940d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:03.660811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a4508-93ac-4aec-bb68-eec4d1e9940d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:03.681520 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-674a4508-93ac-4aec-bb68-eec4d1e9940d None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:03.682786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-674a4508-93ac-4aec-bb68-eec4d1e9940d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:03.690684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1112/2229] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:26:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:03.703835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37f9a0-4514-4508-8fff-2a3c3f10ca31 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:03.716361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37f9a0-4514-4508-8fff-2a3c3f10ca31 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:03.722077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cfea7-d5e9-439c-bb64-d1ca3937c233 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:03.722734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cfea7-d5e9-439c-bb64-d1ca3937c233 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:03.723012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cfea7-d5e9-439c-bb64-d1ca3937c233 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:03.723293 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659cfea7-d5e9-439c-bb64-d1ca3937c233 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:03.732110 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659cfea7-d5e9-439c-bb64-d1ca3937c233 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:03.733731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659cfea7-d5e9-439c-bb64-d1ca3937c233 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:03.742748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1118/2230] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:26:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 16:26:04.147006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37348c9c-c377-49cc-a35f-f255a6e468ff None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:04.147370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37348c9c-c377-49cc-a35f-f255a6e468ff None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:04.148292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37348c9c-c377-49cc-a35f-f255a6e468ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:04.192542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-37348c9c-c377-49cc-a35f-f255a6e468ff None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:04.193773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-37348c9c-c377-49cc-a35f-f255a6e468ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:04.203619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1113/2231] 10.222.0.64 () {58 vars in 967 bytes} [Wed Sep 21 16:26:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:04.217196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cfea7-d5e9-439c-bb64-d1ca3937c233 service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:04.230776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659cfea7-d5e9-439c-bb64-d1ca3937c233 service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:04.253908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394755f2-229d-47dd-a2d0-5fb52ed7f131 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:04.254651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394755f2-229d-47dd-a2d0-5fb52ed7f131 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:04.255025 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394755f2-229d-47dd-a2d0-5fb52ed7f131 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:04.255370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394755f2-229d-47dd-a2d0-5fb52ed7f131 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:04.265415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394755f2-229d-47dd-a2d0-5fb52ed7f131 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:04.267168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394755f2-229d-47dd-a2d0-5fb52ed7f131 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:04.277507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1119/2232] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:26:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 16:26:05.185504 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37348c9c-c377-49cc-a35f-f255a6e468ff None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:05.186355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394755f2-229d-47dd-a2d0-5fb52ed7f131 service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:05.187747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-37348c9c-c377-49cc-a35f-f255a6e468ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:05.190938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-394755f2-229d-47dd-a2d0-5fb52ed7f131 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:05.338727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394755f2-229d-47dd-a2d0-5fb52ed7f131 service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:05.347528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee119bc-88af-42e2-95ea-c7a4dfde21b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:05.348644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee119bc-88af-42e2-95ea-c7a4dfde21b8 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:05.350465 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee119bc-88af-42e2-95ea-c7a4dfde21b8 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:05.351049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee119bc-88af-42e2-95ea-c7a4dfde21b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:05.353363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2ee119bc-88af-42e2-95ea-c7a4dfde21b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:05.371749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:05.386576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee119bc-88af-42e2-95ea-c7a4dfde21b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:05.389493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee119bc-88af-42e2-95ea-c7a4dfde21b8 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:05.412229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37348c9c-c377-49cc-a35f-f255a6e468ff None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:05.422481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06093ac3-a02b-41aa-9f94-6f317ac6f4ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:05.423632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06093ac3-a02b-41aa-9f94-6f317ac6f4ac service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:05.424167 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1120/2233] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:26:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:26:05.426575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06093ac3-a02b-41aa-9f94-6f317ac6f4ac service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:05.427072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06093ac3-a02b-41aa-9f94-6f317ac6f4ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:05.447719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06093ac3-a02b-41aa-9f94-6f317ac6f4ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:05.449501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06093ac3-a02b-41aa-9f94-6f317ac6f4ac service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:05.468945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1114/2234] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:26:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:26:06.034373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71558a5e-2f53-4eda-a0e7-875a33b35823 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:06.034674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71558a5e-2f53-4eda-a0e7-875a33b35823 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:06.035148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71558a5e-2f53-4eda-a0e7-875a33b35823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:06.059534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-71558a5e-2f53-4eda-a0e7-875a33b35823 None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:06.062949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-71558a5e-2f53-4eda-a0e7-875a33b35823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:06.079991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1121/2235] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:26:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:06.102380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06093ac3-a02b-41aa-9f94-6f317ac6f4ac service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:06.124291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06093ac3-a02b-41aa-9f94-6f317ac6f4ac service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:06.130307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e1d41-8052-4f5e-a957-da01671957c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:06.131292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e1d41-8052-4f5e-a957-da01671957c1 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:06.131868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e1d41-8052-4f5e-a957-da01671957c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:06.132357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e1d41-8052-4f5e-a957-da01671957c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:06.144533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e1d41-8052-4f5e-a957-da01671957c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:06.146956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e1d41-8052-4f5e-a957-da01671957c1 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:06.167940 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1115/2236] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:26:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 16:26:06.846107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71558a5e-2f53-4eda-a0e7-875a33b35823 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:06.848022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-71558a5e-2f53-4eda-a0e7-875a33b35823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:06.889454 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:06.970028 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71558a5e-2f53-4eda-a0e7-875a33b35823 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:06.981551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:06.982803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:06.984290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:06.985289 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] PATH_INFO: `/v3/users/22d1610491ba481e80f3e8cccfea3aa1` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:06.988680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22d1610491ba481e80f3e8cccfea3aa1)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:06.991341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:07.087783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22d1610491ba481e80f3e8cccfea3aa1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:07.099514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22d1610491ba481e80f3e8cccfea3aa1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:07.100669 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22d1610491ba481e80f3e8cccfea3aa1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:07.110465 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22d1610491ba481e80f3e8cccfea3aa1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:07.136506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22d1610491ba481e80f3e8cccfea3aa1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:07.147500 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1122/2237] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:26:06 2022] DELETE /identity/v3/users/22d1610491ba481e80f3e8cccfea3aa1 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:26:07.151793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8d8a2-fb60-4a81-8098-54f20a165746 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:07.152226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8d8a2-fb60-4a81-8098-54f20a165746 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:07.152322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8d8a2-fb60-4a81-8098-54f20a165746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:07.176406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b9b8d8a2-fb60-4a81-8098-54f20a165746 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:07.178445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b9b8d8a2-fb60-4a81-8098-54f20a165746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:07.276574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1116/2238] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:07.289476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:07.291045 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:07.410312 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf9cb8-6453-4b73-99a2-871cebb79969 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:07.416060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c39a5-dc15-4259-a186-f36c4e9387b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:07.417281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c39a5-dc15-4259-a186-f36c4e9387b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:07.417786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c39a5-dc15-4259-a186-f36c4e9387b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:07.418324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c39a5-dc15-4259-a186-f36c4e9387b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:07.432015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c39a5-dc15-4259-a186-f36c4e9387b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:07.433955 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c39a5-dc15-4259-a186-f36c4e9387b6 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:07.446030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1123/2239] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:26:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:26:07.683706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8d8a2-fb60-4a81-8098-54f20a165746 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:07.686767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b9b8d8a2-fb60-4a81-8098-54f20a165746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:07.701205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8d8a2-fb60-4a81-8098-54f20a165746 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:07.708811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5aa5f9-edbc-4646-9513-651982f45eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:07.709599 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5aa5f9-edbc-4646-9513-651982f45eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:07.709962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5aa5f9-edbc-4646-9513-651982f45eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:07.710435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5aa5f9-edbc-4646-9513-651982f45eca admin admin] PATH_INFO: `/v3/projects/de5ebe21fe434bc3ad5ab4354c52fcdb` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:07.712143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5aa5f9-edbc-4646-9513-651982f45eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=de5ebe21fe434bc3ad5ab4354c52fcdb)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:07.714769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5aa5f9-edbc-4646-9513-651982f45eca admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:07.782477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe5aa5f9-edbc-4646-9513-651982f45eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de5ebe21fe434bc3ad5ab4354c52fcdb', 'request_id': 'req-fe5aa5f9-edbc-4646-9513-651982f45eca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:07.794352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe5aa5f9-edbc-4646-9513-651982f45eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de5ebe21fe434bc3ad5ab4354c52fcdb', 'request_id': 'req-fe5aa5f9-edbc-4646-9513-651982f45eca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:07.803619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe5aa5f9-edbc-4646-9513-651982f45eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de5ebe21fe434bc3ad5ab4354c52fcdb', 'request_id': 'req-fe5aa5f9-edbc-4646-9513-651982f45eca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:07.821260 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe5aa5f9-edbc-4646-9513-651982f45eca admin admin] The token cache is being invalidate because project de5ebe21fe434bc3ad5ab4354c52fcdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:26:07.821983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe5aa5f9-edbc-4646-9513-651982f45eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:07.823957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe5aa5f9-edbc-4646-9513-651982f45eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:07.828405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1117/2240] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:26:07 2022] DELETE /identity/v3/projects/de5ebe21fe434bc3ad5ab4354c52fcdb => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:26:07.840083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16711b68-b418-4e31-8387-4eb1a8f0aee3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:07.840500 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16711b68-b418-4e31-8387-4eb1a8f0aee3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:07.840844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16711b68-b418-4e31-8387-4eb1a8f0aee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:07.864392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-16711b68-b418-4e31-8387-4eb1a8f0aee3 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:07.866000 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-16711b68-b418-4e31-8387-4eb1a8f0aee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:07.927109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:07.986901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1124/2241] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:26:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:08.002688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e423c4-23f2-437c-bb27-f4ed9ba9a964 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:08.003192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e423c4-23f2-437c-bb27-f4ed9ba9a964 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:08.003694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e423c4-23f2-437c-bb27-f4ed9ba9a964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:08.071675 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a4e423c4-23f2-437c-bb27-f4ed9ba9a964 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:08.078528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a4e423c4-23f2-437c-bb27-f4ed9ba9a964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:08.106101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1118/2242] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:26:08.110976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16711b68-b418-4e31-8387-4eb1a8f0aee3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:08.143389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16711b68-b418-4e31-8387-4eb1a8f0aee3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:08.155299 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8c199-28a1-45d5-a0dd-539d1c28e8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:08.157893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8c199-28a1-45d5-a0dd-539d1c28e8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:08.158783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8c199-28a1-45d5-a0dd-539d1c28e8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:08.159227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8c199-28a1-45d5-a0dd-539d1c28e8d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:08.161836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8c199-28a1-45d5-a0dd-539d1c28e8d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:26:08.161836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8c199-28a1-45d5-a0dd-539d1c28e8d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:08.175849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a8c199-28a1-45d5-a0dd-539d1c28e8d9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:08.236459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1125/2243] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:26:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:26:08.240505 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e423c4-23f2-437c-bb27-f4ed9ba9a964 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:08.253852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e423c4-23f2-437c-bb27-f4ed9ba9a964 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:08.262804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1db0c4-ffa3-42b5-8b31-95c235483670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:08.263438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1db0c4-ffa3-42b5-8b31-95c235483670 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:08.263721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1db0c4-ffa3-42b5-8b31-95c235483670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:08.264008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1db0c4-ffa3-42b5-8b31-95c235483670 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:08.265296 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1db0c4-ffa3-42b5-8b31-95c235483670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:08.268785 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1db0c4-ffa3-42b5-8b31-95c235483670 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:08.451834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1119/2244] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:26:08 2022] POST /identity/v3/projects => generated 379 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:26:08.457859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8c199-28a1-45d5-a0dd-539d1c28e8d9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:08.491856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8c199-28a1-45d5-a0dd-539d1c28e8d9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:08.671910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6e923-5d82-4e70-8b7b-523edf87ed5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:08.672648 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6e923-5d82-4e70-8b7b-523edf87ed5d admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:08.673204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6e923-5d82-4e70-8b7b-523edf87ed5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:08.674135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6e923-5d82-4e70-8b7b-523edf87ed5d admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:08.674932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6e923-5d82-4e70-8b7b-523edf87ed5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:08.677276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6e923-5d82-4e70-8b7b-523edf87ed5d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:08.707852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1126/2245] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:26:08 2022] POST /identity/v3/users => generated 290 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:26:08.719214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1db0c4-ffa3-42b5-8b31-95c235483670 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:08.744684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1db0c4-ffa3-42b5-8b31-95c235483670 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:08.769633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326c220-bfc2-4cbc-b164-0322f724fc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:08.770302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326c220-bfc2-4cbc-b164-0322f724fc1a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:08.770655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326c220-bfc2-4cbc-b164-0322f724fc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:08.770867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326c220-bfc2-4cbc-b164-0322f724fc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:08.771568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326c220-bfc2-4cbc-b164-0322f724fc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:08.773574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326c220-bfc2-4cbc-b164-0322f724fc1a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:08.826250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1120/2246] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:26:08.836877 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6e923-5d82-4e70-8b7b-523edf87ed5d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:08.892560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6e923-5d82-4e70-8b7b-523edf87ed5d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:08.915749 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aedc58-4edf-481b-b2fd-f578e796161a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:08.916644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aedc58-4edf-481b-b2fd-f578e796161a admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:08.916960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aedc58-4edf-481b-b2fd-f578e796161a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:08.917338 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aedc58-4edf-481b-b2fd-f578e796161a admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:08.918031 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aedc58-4edf-481b-b2fd-f578e796161a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:08.919900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aedc58-4edf-481b-b2fd-f578e796161a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:08.941975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1127/2247] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:26:08.942820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326c220-bfc2-4cbc-b164-0322f724fc1a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:08.964728 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326c220-bfc2-4cbc-b164-0322f724fc1a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:08.973710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1b6b8-af9b-49c9-b98b-956e80fd65df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:08.974425 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1b6b8-af9b-49c9-b98b-956e80fd65df admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:08.974805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1b6b8-af9b-49c9-b98b-956e80fd65df admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:08.975073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1b6b8-af9b-49c9-b98b-956e80fd65df admin admin] PATH_INFO: `/v3/projects/d529e1f3de6944f881c7fc572b1dc78e/users/f619261c21204ea3b5ba624177eb8747/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:08.995289 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:09.013171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1b6b8-af9b-49c9-b98b-956e80fd65df admin admin] RBAC: Authorizing `identity:create_grant(project_id=d529e1f3de6944f881c7fc572b1dc78e, user_id=f619261c21204ea3b5ba624177eb8747, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:09.019071 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1b6b8-af9b-49c9-b98b-956e80fd65df admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:09.047725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1121/2248] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:26:08 2022] PUT /identity/v3/projects/d529e1f3de6944f881c7fc572b1dc78e/users/f619261c21204ea3b5ba624177eb8747/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:26:09.052790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dc259-f72b-4ab9-8070-5373bcfc8393 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:09.053575 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dc259-f72b-4ab9-8070-5373bcfc8393 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:09.054327 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dc259-f72b-4ab9-8070-5373bcfc8393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:09.091825 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5c3dc259-f72b-4ab9-8070-5373bcfc8393 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:09.096249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5c3dc259-f72b-4ab9-8070-5373bcfc8393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:09.279926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1128/2249] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:26:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:09.328148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1b6b8-af9b-49c9-b98b-956e80fd65df admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:09.333115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1db1b6b8-af9b-49c9-b98b-956e80fd65df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:09.465096 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1b6b8-af9b-49c9-b98b-956e80fd65df admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:09.470576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025043d6-64a1-4b76-ab90-bf1d59e4fdc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:09.471618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025043d6-64a1-4b76-ab90-bf1d59e4fdc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:09.472627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025043d6-64a1-4b76-ab90-bf1d59e4fdc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:09.472921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025043d6-64a1-4b76-ab90-bf1d59e4fdc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:09.493026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025043d6-64a1-4b76-ab90-bf1d59e4fdc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:09.495430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025043d6-64a1-4b76-ab90-bf1d59e4fdc5 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:09.520594 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1122/2250] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:26:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:26:11.551197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e5a9e-9944-4920-846e-0938830fd293 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:11.552683 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e5a9e-9944-4920-846e-0938830fd293 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:11.553446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e5a9e-9944-4920-846e-0938830fd293 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:11.556097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1129/2251] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:26:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 21 16:26:11.601394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815ecef-09dd-43e4-b437-fdedce5ba81e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:11.602392 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815ecef-09dd-43e4-b437-fdedce5ba81e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:11.603095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815ecef-09dd-43e4-b437-fdedce5ba81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:11.615210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa334016-7aef-493a-bfe6-8e59a7c0944c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:11.615959 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa334016-7aef-493a-bfe6-8e59a7c0944c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:11.616450 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa334016-7aef-493a-bfe6-8e59a7c0944c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:11.676814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-aa334016-7aef-493a-bfe6-8e59a7c0944c None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:11.681241 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-aa334016-7aef-493a-bfe6-8e59a7c0944c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:11.756016 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c815ecef-09dd-43e4-b437-fdedce5ba81e None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:11.758041 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c815ecef-09dd-43e4-b437-fdedce5ba81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:12.008444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:12.086528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1130/2252] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:26:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:12.102596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa334016-7aef-493a-bfe6-8e59a7c0944c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:12.104275 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-aa334016-7aef-493a-bfe6-8e59a7c0944c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:12.179972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1123/2253] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:26:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 578 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:12.294790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa334016-7aef-493a-bfe6-8e59a7c0944c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:12.300677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921fbb9-440e-417a-8b44-3f37db293ac5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:12.301324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921fbb9-440e-417a-8b44-3f37db293ac5 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:12.301608 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921fbb9-440e-417a-8b44-3f37db293ac5 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:12.301882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921fbb9-440e-417a-8b44-3f37db293ac5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:12.315862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4921fbb9-440e-417a-8b44-3f37db293ac5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:12.319108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4921fbb9-440e-417a-8b44-3f37db293ac5 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:12.348448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1131/2254] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:26:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 16:26:22.137092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=60,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:22.329842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=35 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:24.192690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91333e9a-d866-4f66-bbe3-a07fcc95176e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:24.193342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91333e9a-d866-4f66-bbe3-a07fcc95176e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:24.193438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91333e9a-d866-4f66-bbe3-a07fcc95176e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:24.296639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-91333e9a-d866-4f66-bbe3-a07fcc95176e None None] MFA Rules not processed for user `f619261c21204ea3b5ba624177eb8747`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:24.299320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-91333e9a-d866-4f66-bbe3-a07fcc95176e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:24.472600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1124/2255] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:26:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:24.478162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921fbb9-440e-417a-8b44-3f37db293ac5 service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:24.497080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921fbb9-440e-417a-8b44-3f37db293ac5 service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:24.503900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76dcac-39e5-4336-b3a4-46c39825db48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:24.510791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76dcac-39e5-4336-b3a4-46c39825db48 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:24.511288 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76dcac-39e5-4336-b3a4-46c39825db48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:24.511721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76dcac-39e5-4336-b3a4-46c39825db48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:24.512621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76dcac-39e5-4336-b3a4-46c39825db48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:24.515485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76dcac-39e5-4336-b3a4-46c39825db48 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:24.687529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1132/2256] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:26:24 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:26:24.688762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91333e9a-d866-4f66-bbe3-a07fcc95176e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:24.725566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91333e9a-d866-4f66-bbe3-a07fcc95176e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:25.121175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf340a62-7863-4825-8d29-f960df106820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:25.121993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf340a62-7863-4825-8d29-f960df106820 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:25.122308 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf340a62-7863-4825-8d29-f960df106820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:25.123459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf340a62-7863-4825-8d29-f960df106820 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:25.123872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf340a62-7863-4825-8d29-f960df106820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:25.147100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf340a62-7863-4825-8d29-f960df106820 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:25.224149 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1125/2257] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:26:24 2022] POST /identity/v3/users => generated 296 bytes in 536 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:26:25.228554 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76dcac-39e5-4336-b3a4-46c39825db48 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:25.245742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76dcac-39e5-4336-b3a4-46c39825db48 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:25.270247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1fccd-85c9-44dc-85d1-edba00283152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:25.270887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1fccd-85c9-44dc-85d1-edba00283152 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:25.271172 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1fccd-85c9-44dc-85d1-edba00283152 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:25.271451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1fccd-85c9-44dc-85d1-edba00283152 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:25.272128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1fccd-85c9-44dc-85d1-edba00283152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:25.273794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1fccd-85c9-44dc-85d1-edba00283152 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:25.355121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1133/2258] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:26:25.359527 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf340a62-7863-4825-8d29-f960df106820 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:25.382159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf340a62-7863-4825-8d29-f960df106820 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:25.400059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdebc6-597c-4bf7-aa3c-5720077345c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:25.400904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdebc6-597c-4bf7-aa3c-5720077345c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:25.401577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdebc6-597c-4bf7-aa3c-5720077345c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:25.401977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdebc6-597c-4bf7-aa3c-5720077345c9 admin admin] PATH_INFO: `/v3/projects/70ec56eccacb4e278f80206e651d397c/users/3bb9ac809b2e43db922954607ff738bb/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:25.448860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdebc6-597c-4bf7-aa3c-5720077345c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70ec56eccacb4e278f80206e651d397c, user_id=3bb9ac809b2e43db922954607ff738bb, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:25.453901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdebc6-597c-4bf7-aa3c-5720077345c9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:25.521034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1126/2259] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:26:25 2022] PUT /identity/v3/projects/70ec56eccacb4e278f80206e651d397c/users/3bb9ac809b2e43db922954607ff738bb/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:26:25.522515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1fccd-85c9-44dc-85d1-edba00283152 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:25.547424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1fccd-85c9-44dc-85d1-edba00283152 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:25.905722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce820c7-c509-4a1e-90e4-eec0c03ecfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:25.906404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce820c7-c509-4a1e-90e4-eec0c03ecfef admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:25.911007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce820c7-c509-4a1e-90e4-eec0c03ecfef admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:25.911007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce820c7-c509-4a1e-90e4-eec0c03ecfef admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:25.911007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce820c7-c509-4a1e-90e4-eec0c03ecfef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:25.912638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce820c7-c509-4a1e-90e4-eec0c03ecfef admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:25.983854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1134/2260] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:26:25.985671 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdebc6-597c-4bf7-aa3c-5720077345c9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:26.034302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdebc6-597c-4bf7-aa3c-5720077345c9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:26.059548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48de446-0a35-4ad4-9f92-d3865fe6de12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:26.061209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48de446-0a35-4ad4-9f92-d3865fe6de12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:26.061661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48de446-0a35-4ad4-9f92-d3865fe6de12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:26.073945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48de446-0a35-4ad4-9f92-d3865fe6de12 admin admin] PATH_INFO: `/v3/projects/70ec56eccacb4e278f80206e651d397c/users/3bb9ac809b2e43db922954607ff738bb/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:26.084543 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48de446-0a35-4ad4-9f92-d3865fe6de12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70ec56eccacb4e278f80206e651d397c, user_id=3bb9ac809b2e43db922954607ff738bb, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:26.106287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48de446-0a35-4ad4-9f92-d3865fe6de12 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:26.147628 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1127/2261] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:26:25 2022] PUT /identity/v3/projects/70ec56eccacb4e278f80206e651d397c/users/3bb9ac809b2e43db922954607ff738bb/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:26:33.169357 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce820c7-c509-4a1e-90e4-eec0c03ecfef admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:33.171242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0ce820c7-c509-4a1e-90e4-eec0c03ecfef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:33.295798 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce820c7-c509-4a1e-90e4-eec0c03ecfef admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:33.367873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bdc7c-dcd3-4e85-9bc1-f0ff42a4c5f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2d76fb9cbdd40f9b2a748209cee5b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:33.369622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bdc7c-dcd3-4e85-9bc1-f0ff42a4c5f1 service glance] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:33.370610 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bdc7c-dcd3-4e85-9bc1-f0ff42a4c5f1 service glance] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:33.371400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bdc7c-dcd3-4e85-9bc1-f0ff42a4c5f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:33.463262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:33.527852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bdc7c-dcd3-4e85-9bc1-f0ff42a4c5f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:33.530841 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bdc7c-dcd3-4e85-9bc1-f0ff42a4c5f1 service glance] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:33.562775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1135/2262] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:26:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:26:33.714528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ddcd5-132d-4cb0-83e3-cde104b97d71 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:33.715173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ddcd5-132d-4cb0-83e3-cde104b97d71 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:33.715173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ddcd5-132d-4cb0-83e3-cde104b97d71 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:33.717304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1128/2263] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:26:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:26:33.724137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e7524-0386-4220-b9ed-ed15f9555b6d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:33.724772 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e7524-0386-4220-b9ed-ed15f9555b6d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:33.725322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e7524-0386-4220-b9ed-ed15f9555b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:33.764574 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-fe3e7524-0386-4220-b9ed-ed15f9555b6d None None] MFA Rules not processed for user `e175b8de037d45d48cbddd64e552f6f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:33.767536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fe3e7524-0386-4220-b9ed-ed15f9555b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:33.917473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1136/2264] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:26:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:33.942939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ddcd5-132d-4cb0-83e3-cde104b97d71 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:33.944678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ee5ddcd5-132d-4cb0-83e3-cde104b97d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:33.969531 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ddcd5-132d-4cb0-83e3-cde104b97d71 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:33.976269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5252a6-b9a4-418d-a71a-c2e8d2db47cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:33.977320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5252a6-b9a4-418d-a71a-c2e8d2db47cf service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:33.982351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5252a6-b9a4-418d-a71a-c2e8d2db47cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:33.982977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5252a6-b9a4-418d-a71a-c2e8d2db47cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:34.003139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5252a6-b9a4-418d-a71a-c2e8d2db47cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:34.004986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5252a6-b9a4-418d-a71a-c2e8d2db47cf service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:34.032912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1129/2265] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:26:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:26:34.182153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f87e-3f01-4261-aad3-840024a2298a None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:34.182708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f87e-3f01-4261-aad3-840024a2298a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:34.182708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3977f87e-3f01-4261-aad3-840024a2298a None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:34.184019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1137/2266] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:26:34.194878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf65dd9-b4bf-4a48-8c4d-3be8e367ec43 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:34.195170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf65dd9-b4bf-4a48-8c4d-3be8e367ec43 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:34.195450 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf65dd9-b4bf-4a48-8c4d-3be8e367ec43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:34.237850 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ecf65dd9-b4bf-4a48-8c4d-3be8e367ec43 None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:34.239444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ecf65dd9-b4bf-4a48-8c4d-3be8e367ec43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:34.272068 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1130/2267] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:26:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:34.278257 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977f87e-3f01-4261-aad3-840024a2298a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:34.291509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3977f87e-3f01-4261-aad3-840024a2298a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:34.295838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9d689-d247-4b67-ae35-c5a25bfafa62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:34.296460 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9d689-d247-4b67-ae35-c5a25bfafa62 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:34.297702 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9d689-d247-4b67-ae35-c5a25bfafa62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:34.297991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9d689-d247-4b67-ae35-c5a25bfafa62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:34.313854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9d689-d247-4b67-ae35-c5a25bfafa62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:34.316769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b9d689-d247-4b67-ae35-c5a25bfafa62 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:34.346917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1138/2268] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:26:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:26:41.060107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365449a2-3c46-4637-998a-40824aef9682 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:41.062611 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365449a2-3c46-4637-998a-40824aef9682 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:41.063383 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365449a2-3c46-4637-998a-40824aef9682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:41.161255 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-365449a2-3c46-4637-998a-40824aef9682 None None] MFA Rules not processed for user `3bb9ac809b2e43db922954607ff738bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:41.162850 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-365449a2-3c46-4637-998a-40824aef9682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:41.175865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:41.325930 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1131/2269] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:26:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:41.333057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9d689-d247-4b67-ae35-c5a25bfafa62 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:41.335596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d8b9d689-d247-4b67-ae35-c5a25bfafa62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:41.361278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9d689-d247-4b67-ae35-c5a25bfafa62 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:41.367907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f841e-91b2-4633-b1e2-94ba400f5bb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:41.368537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f841e-91b2-4633-b1e2-94ba400f5bb7 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:41.368808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f841e-91b2-4633-b1e2-94ba400f5bb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:41.369311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f841e-91b2-4633-b1e2-94ba400f5bb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:41.497044 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f841e-91b2-4633-b1e2-94ba400f5bb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:41.499327 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f841e-91b2-4633-b1e2-94ba400f5bb7 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:41.530559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1139/2270] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:26:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:26:42.697720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38392b8d-ec72-430f-9365-c414abe9865b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:42.698381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38392b8d-ec72-430f-9365-c414abe9865b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:42.698786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38392b8d-ec72-430f-9365-c414abe9865b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:42.792919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-38392b8d-ec72-430f-9365-c414abe9865b None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:42.797228 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-38392b8d-ec72-430f-9365-c414abe9865b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:42.819778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad42528-2674-47f7-a119-6b422697ecfe None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:42.820263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad42528-2674-47f7-a119-6b422697ecfe None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:42.820616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad42528-2674-47f7-a119-6b422697ecfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:42.874148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3ad42528-2674-47f7-a119-6b422697ecfe None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:26:42.875805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3ad42528-2674-47f7-a119-6b422697ecfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:42.941304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1132/2271] 10.222.0.64 () {58 vars in 967 bytes} [Wed Sep 21 16:26:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:26:42.956477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38392b8d-ec72-430f-9365-c414abe9865b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:42.976007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:42.998691 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1140/2272] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:26:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:26:43.026786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad42528-2674-47f7-a119-6b422697ecfe None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:43.186223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38392b8d-ec72-430f-9365-c414abe9865b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:43.201313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a3ff2-1226-4e6b-af4e-06bdf1174bc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:43.203654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a3ff2-1226-4e6b-af4e-06bdf1174bc7 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:43.204532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a3ff2-1226-4e6b-af4e-06bdf1174bc7 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:43.205604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a3ff2-1226-4e6b-af4e-06bdf1174bc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:43.229894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a3ff2-1226-4e6b-af4e-06bdf1174bc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:43.232926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a3ff2-1226-4e6b-af4e-06bdf1174bc7 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:43.261523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad42528-2674-47f7-a119-6b422697ecfe None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:43.265497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1133/2273] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:26:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 16:26:43.273983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5314b8-c989-407a-84ed-5410cb267be9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:43.274749 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5314b8-c989-407a-84ed-5410cb267be9 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:43.275096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5314b8-c989-407a-84ed-5410cb267be9 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:43.275587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5314b8-c989-407a-84ed-5410cb267be9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:43.286187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5314b8-c989-407a-84ed-5410cb267be9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:43.304399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5314b8-c989-407a-84ed-5410cb267be9 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:43.324897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1141/2274] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:26:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 16:26:47.797425 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a3ff2-1226-4e6b-af4e-06bdf1174bc7 service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:47.799988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-010a3ff2-1226-4e6b-af4e-06bdf1174bc7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:26:47.980071 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:48.039683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a3ff2-1226-4e6b-af4e-06bdf1174bc7 service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:48.046975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:48.048448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:48.048854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:48.049309 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] PATH_INFO: `/v3/users/415e8845ba8d46cc9dc939ef4556e865` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:48.051072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] RBAC: Authorizing `identity:delete_user(user_id=415e8845ba8d46cc9dc939ef4556e865)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:48.053987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:48.266187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '415e8845ba8d46cc9dc939ef4556e865'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:48.282549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '415e8845ba8d46cc9dc939ef4556e865'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:48.308768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '415e8845ba8d46cc9dc939ef4556e865'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:48.309826 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '415e8845ba8d46cc9dc939ef4556e865'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:48.325221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '415e8845ba8d46cc9dc939ef4556e865'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:48.339733 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1134/2275] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:26:47 2022] DELETE /identity/v3/users/415e8845ba8d46cc9dc939ef4556e865 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:26:49.075207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5314b8-c989-407a-84ed-5410cb267be9 service swift] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:26:49.094910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5314b8-c989-407a-84ed-5410cb267be9 service swift] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:26:49.455883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4296cb2b-e2a7-4575-80b4-2e621b0661c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:26:49.456836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296cb2b-e2a7-4575-80b4-2e621b0661c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:26:49.457172 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296cb2b-e2a7-4575-80b4-2e621b0661c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:26:49.457726 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4296cb2b-e2a7-4575-80b4-2e621b0661c2 admin admin] PATH_INFO: `/v3/projects/97c0b8a086b046ddb2b2d475124dc692` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:26:49.472180 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296cb2b-e2a7-4575-80b4-2e621b0661c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97c0b8a086b046ddb2b2d475124dc692)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:26:49.475088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4296cb2b-e2a7-4575-80b4-2e621b0661c2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:26:49.736632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4296cb2b-e2a7-4575-80b4-2e621b0661c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97c0b8a086b046ddb2b2d475124dc692', 'request_id': 'req-4296cb2b-e2a7-4575-80b4-2e621b0661c2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:49.784848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4296cb2b-e2a7-4575-80b4-2e621b0661c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97c0b8a086b046ddb2b2d475124dc692', 'request_id': 'req-4296cb2b-e2a7-4575-80b4-2e621b0661c2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:49.797722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4296cb2b-e2a7-4575-80b4-2e621b0661c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97c0b8a086b046ddb2b2d475124dc692', 'request_id': 'req-4296cb2b-e2a7-4575-80b4-2e621b0661c2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:49.845708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4296cb2b-e2a7-4575-80b4-2e621b0661c2 admin admin] The token cache is being invalidate because project 97c0b8a086b046ddb2b2d475124dc692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:26:49.846620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4296cb2b-e2a7-4575-80b4-2e621b0661c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:49.860358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4296cb2b-e2a7-4575-80b4-2e621b0661c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:26:49.897046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1142/2276] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:26:49 2022] DELETE /identity/v3/projects/97c0b8a086b046ddb2b2d475124dc692 => generated 0 bytes in 808 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:26:58.328893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=34,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:26:59.823062 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:27:01.850204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:27:01.852263 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:27:01.926945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4b5d2-a30c-4c23-bd7a-c1433330d767 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:27:02.001213 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f320e862-ec3f-4954-83e9-cdca841d7a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:27:02.001842 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320e862-ec3f-4954-83e9-cdca841d7a09 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:27:02.002206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320e862-ec3f-4954-83e9-cdca841d7a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:27:02.002439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f320e862-ec3f-4954-83e9-cdca841d7a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:27:02.004389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f320e862-ec3f-4954-83e9-cdca841d7a09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:27:02.088783 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320e862-ec3f-4954-83e9-cdca841d7a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:27:02.091405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f320e862-ec3f-4954-83e9-cdca841d7a09 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:27:02.110435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1135/2277] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:27:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:27:04.381036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda2197-96a3-430b-84f9-f547636a3d67 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:27:04.381036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda2197-96a3-430b-84f9-f547636a3d67 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:27:04.381684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda2197-96a3-430b-84f9-f547636a3d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:27:04.411499 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9cda2197-96a3-430b-84f9-f547636a3d67 None None] MFA Rules not processed for user `f619261c21204ea3b5ba624177eb8747`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:27:04.413622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9cda2197-96a3-430b-84f9-f547636a3d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:27:04.534723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1143/2278] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:27:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:27:07.085868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a1c6c-8a45-4879-94fd-36ec1b237893 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:27:07.086744 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a1c6c-8a45-4879-94fd-36ec1b237893 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:27:07.087013 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a1c6c-8a45-4879-94fd-36ec1b237893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:27:07.120522 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-351a1c6c-8a45-4879-94fd-36ec1b237893 None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:27:07.123195 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-351a1c6c-8a45-4879-94fd-36ec1b237893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:27:07.214228 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1136/2279] 10.222.0.64 () {58 vars in 967 bytes} [Wed Sep 21 16:27:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:27:07.230977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda2197-96a3-430b-84f9-f547636a3d67 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:27:07.234811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9cda2197-96a3-430b-84f9-f547636a3d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:27:07.361685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda2197-96a3-430b-84f9-f547636a3d67 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:27:07.367345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade85db-7f91-4317-9670-b8fcc0b4598f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:27:07.368420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade85db-7f91-4317-9670-b8fcc0b4598f service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:27:07.368898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade85db-7f91-4317-9670-b8fcc0b4598f service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:27:07.369355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade85db-7f91-4317-9670-b8fcc0b4598f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:27:07.381658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade85db-7f91-4317-9670-b8fcc0b4598f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:27:07.384546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade85db-7f91-4317-9670-b8fcc0b4598f service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:27:07.396515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1144/2280] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:27:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 16:27:17.201655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:27:17.392977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=57 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:27:26.442608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a1c6c-8a45-4879-94fd-36ec1b237893 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:27:26.445075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-351a1c6c-8a45-4879-94fd-36ec1b237893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:27:26.457810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a1c6c-8a45-4879-94fd-36ec1b237893 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:27:26.461427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749e4ea-134a-4390-9fef-0d472e762f00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:27:26.462682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749e4ea-134a-4390-9fef-0d472e762f00 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:27:26.462960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749e4ea-134a-4390-9fef-0d472e762f00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:27:26.463230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749e4ea-134a-4390-9fef-0d472e762f00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:27:26.464956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1749e4ea-134a-4390-9fef-0d472e762f00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:27:26.556075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749e4ea-134a-4390-9fef-0d472e762f00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:27:26.557521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749e4ea-134a-4390-9fef-0d472e762f00 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:27:26.567156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1137/2281] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:27:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:27:36.562898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=30 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:27:47.598827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb8150-fba4-4ecc-a9d1-46fb7d629ef7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:27:47.599378 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb8150-fba4-4ecc-a9d1-46fb7d629ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:27:47.599378 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb8150-fba4-4ecc-a9d1-46fb7d629ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:27:47.619406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-98cb8150-fba4-4ecc-a9d1-46fb7d629ef7 None None] MFA Rules not processed for user `b917d84a650f4355a676e315ea039f0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:27:47.620693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-98cb8150-fba4-4ecc-a9d1-46fb7d629ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:27:47.676203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1145/2282] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:27:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:27:51.941841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749e4ea-134a-4390-9fef-0d472e762f00 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:27:51.951467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749e4ea-134a-4390-9fef-0d472e762f00 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:27:51.955395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0323081-1b85-4cb0-92f1-c392b0307607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:27:51.956004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0323081-1b85-4cb0-92f1-c392b0307607 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:27:51.956572 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0323081-1b85-4cb0-92f1-c392b0307607 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:27:51.957046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0323081-1b85-4cb0-92f1-c392b0307607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:27:51.959230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a0323081-1b85-4cb0-92f1-c392b0307607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:27:52.042025 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0323081-1b85-4cb0-92f1-c392b0307607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:27:52.043490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0323081-1b85-4cb0-92f1-c392b0307607 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:27:52.054329 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1138/2283] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:27:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:27:52.512575 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb8150-fba4-4ecc-a9d1-46fb7d629ef7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:27:52.513977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-98cb8150-fba4-4ecc-a9d1-46fb7d629ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:27:52.525863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb8150-fba4-4ecc-a9d1-46fb7d629ef7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:27:52.529387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa181ac-bea8-472d-b52d-3f8851e4ea68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:27:52.529982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa181ac-bea8-472d-b52d-3f8851e4ea68 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:27:52.530291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa181ac-bea8-472d-b52d-3f8851e4ea68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:27:52.530568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa181ac-bea8-472d-b52d-3f8851e4ea68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:27:52.532661 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5fa181ac-bea8-472d-b52d-3f8851e4ea68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:27:52.546295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa181ac-bea8-472d-b52d-3f8851e4ea68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:27:52.547710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa181ac-bea8-472d-b52d-3f8851e4ea68 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:27:52.559511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1146/2284] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:27:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:28:02.049484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=34 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:28:02.554834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=31 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:28:06.092600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdd7fe-bb86-4932-84cd-736d58c8f620 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:06.093160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdd7fe-bb86-4932-84cd-736d58c8f620 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:06.093160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdd7fe-bb86-4932-84cd-736d58c8f620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:06.112879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9bbdd7fe-bb86-4932-84cd-736d58c8f620 None None] MFA Rules not processed for user `d2382a6610ae42fa865283ed12dd9982`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:28:06.114118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9bbdd7fe-bb86-4932-84cd-736d58c8f620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:06.171868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1139/2285] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:28:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:28:06.486194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa181ac-bea8-472d-b52d-3f8851e4ea68 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:28:06.498671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa181ac-bea8-472d-b52d-3f8851e4ea68 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:28:06.505276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ff677-3dd2-4a00-98da-5e58862debec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:28:06.512371 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ff677-3dd2-4a00-98da-5e58862debec service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:06.515629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ff677-3dd2-4a00-98da-5e58862debec service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:06.515779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ff677-3dd2-4a00-98da-5e58862debec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:06.521602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1c5ff677-3dd2-4a00-98da-5e58862debec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:06.630233 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdd7fe-bb86-4932-84cd-736d58c8f620 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:28:06.631692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9bbdd7fe-bb86-4932-84cd-736d58c8f620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:06.665710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdd7fe-bb86-4932-84cd-736d58c8f620 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:28:06.668738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ff677-3dd2-4a00-98da-5e58862debec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:28:06.671737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ff677-3dd2-4a00-98da-5e58862debec service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:28:06.674397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44ec19-1277-43bb-892b-25220e5d4994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:28:06.675022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ec19-1277-43bb-892b-25220e5d4994 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:06.675296 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ec19-1277-43bb-892b-25220e5d4994 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:06.675564 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ec19-1277-43bb-892b-25220e5d4994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:06.695546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1147/2286] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:28:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:28:06.699358 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44ec19-1277-43bb-892b-25220e5d4994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:28:06.703277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44ec19-1277-43bb-892b-25220e5d4994 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:28:06.721489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1140/2287] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:28:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:28:08.786217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c96a5-3521-470a-b3f1-1a2fa205137a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:08.786673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c96a5-3521-470a-b3f1-1a2fa205137a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:08.786753 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c96a5-3521-470a-b3f1-1a2fa205137a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:08.809283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-055c96a5-3521-470a-b3f1-1a2fa205137a None None] MFA Rules not processed for user `2e5204475ce7477f9cfeeffed457a445`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:28:08.810549 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-055c96a5-3521-470a-b3f1-1a2fa205137a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:08.875411 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1148/2288] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:28:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:28:16.540933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44ec19-1277-43bb-892b-25220e5d4994 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:28:16.542602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3e44ec19-1277-43bb-892b-25220e5d4994 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:16.557314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44ec19-1277-43bb-892b-25220e5d4994 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:28:16.563114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c210b01-b133-4d0c-9fd8-29e310076578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:28:16.563808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c210b01-b133-4d0c-9fd8-29e310076578 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:16.564150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c210b01-b133-4d0c-9fd8-29e310076578 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:16.564488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c210b01-b133-4d0c-9fd8-29e310076578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:16.566242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7c210b01-b133-4d0c-9fd8-29e310076578 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:16.577507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c210b01-b133-4d0c-9fd8-29e310076578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:28:16.579457 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c210b01-b133-4d0c-9fd8-29e310076578 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:28:16.589568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1141/2289] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:28:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:28:17.520163 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257add49-329e-44ec-9540-91d85306d58b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:17.520687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257add49-329e-44ec-9540-91d85306d58b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:17.520812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257add49-329e-44ec-9540-91d85306d58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:17.541688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-257add49-329e-44ec-9540-91d85306d58b None None] MFA Rules not processed for user `617ba5f9896b457f8182f760ed44aa58`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:28:17.542956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-257add49-329e-44ec-9540-91d85306d58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:17.550355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1149/2290] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:28:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:28:17.879480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c210b01-b133-4d0c-9fd8-29e310076578 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:28:17.889330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c210b01-b133-4d0c-9fd8-29e310076578 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:28:17.893022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9ffa1-dbf1-42f9-976a-937b94d20ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:28:17.893633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9ffa1-dbf1-42f9-976a-937b94d20ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:17.893920 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9ffa1-dbf1-42f9-976a-937b94d20ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:17.894239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf9ffa1-dbf1-42f9-976a-937b94d20ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:17.896161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dbf9ffa1-dbf1-42f9-976a-937b94d20ca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:17.906761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf9ffa1-dbf1-42f9-976a-937b94d20ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:28:17.908196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf9ffa1-dbf1-42f9-976a-937b94d20ca9 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:28:17.916763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1142/2291] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:28:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:28:18.233904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257add49-329e-44ec-9540-91d85306d58b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:28:18.235423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-257add49-329e-44ec-9540-91d85306d58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:18.314000 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257add49-329e-44ec-9540-91d85306d58b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:28:18.319462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35771aee-f0d2-48b2-b1b1-7e0dd0e62937 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:28:18.320060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35771aee-f0d2-48b2-b1b1-7e0dd0e62937 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:18.320334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35771aee-f0d2-48b2-b1b1-7e0dd0e62937 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:18.320602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35771aee-f0d2-48b2-b1b1-7e0dd0e62937 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:18.322368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-35771aee-f0d2-48b2-b1b1-7e0dd0e62937 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:18.334351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35771aee-f0d2-48b2-b1b1-7e0dd0e62937 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:28:18.336076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35771aee-f0d2-48b2-b1b1-7e0dd0e62937 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:28:18.349168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1150/2292] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:28:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:28:26.466467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9ffa1-dbf1-42f9-976a-937b94d20ca9 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:28:26.475935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf9ffa1-dbf1-42f9-976a-937b94d20ca9 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:28:26.480166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3064d-3f6b-4acb-ad6d-440ca9453bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:28:26.481218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3064d-3f6b-4acb-ad6d-440ca9453bfd service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:26.481718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3064d-3f6b-4acb-ad6d-440ca9453bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:26.482229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3064d-3f6b-4acb-ad6d-440ca9453bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:26.484391 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-62d3064d-3f6b-4acb-ad6d-440ca9453bfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:26.496808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3064d-3f6b-4acb-ad6d-440ca9453bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:28:26.498643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3064d-3f6b-4acb-ad6d-440ca9453bfd service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:28:26.509194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1143/2293] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:28:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:28:28.341925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:28:36.503930 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=49 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:28:45.250932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35771aee-f0d2-48b2-b1b1-7e0dd0e62937 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:28:45.266479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35771aee-f0d2-48b2-b1b1-7e0dd0e62937 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:28:45.272853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fff5fd-dbbc-41f5-91ba-4b9d2b947073 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:28:45.273797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fff5fd-dbbc-41f5-91ba-4b9d2b947073 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:45.274346 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fff5fd-dbbc-41f5-91ba-4b9d2b947073 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:45.275033 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fff5fd-dbbc-41f5-91ba-4b9d2b947073 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:45.277940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f2fff5fd-dbbc-41f5-91ba-4b9d2b947073 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:45.296146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fff5fd-dbbc-41f5-91ba-4b9d2b947073 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:28:45.297921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fff5fd-dbbc-41f5-91ba-4b9d2b947073 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:28:45.311836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1151/2294] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:28:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:28:46.318688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3064d-3f6b-4acb-ad6d-440ca9453bfd service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:28:46.338620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3064d-3f6b-4acb-ad6d-440ca9453bfd service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:28:46.344669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c957d147-747f-40be-8305-b2addb548943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:28:46.345331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957d147-747f-40be-8305-b2addb548943 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:46.345686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957d147-747f-40be-8305-b2addb548943 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:46.345901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c957d147-747f-40be-8305-b2addb548943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:46.350542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c957d147-747f-40be-8305-b2addb548943 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:46.364306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957d147-747f-40be-8305-b2addb548943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:28:46.365907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c957d147-747f-40be-8305-b2addb548943 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:28:46.378310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1144/2295] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:28:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:28:49.294639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fff5fd-dbbc-41f5-91ba-4b9d2b947073 service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:28:49.309467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fff5fd-dbbc-41f5-91ba-4b9d2b947073 service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:28:49.317971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483074e6-5cca-481c-aeb6-cac0b1a4f6bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:28:49.319104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483074e6-5cca-481c-aeb6-cac0b1a4f6bb service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:49.319567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483074e6-5cca-481c-aeb6-cac0b1a4f6bb service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:49.319995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483074e6-5cca-481c-aeb6-cac0b1a4f6bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:49.322022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-483074e6-5cca-481c-aeb6-cac0b1a4f6bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:49.333710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483074e6-5cca-481c-aeb6-cac0b1a4f6bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:28:49.335364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483074e6-5cca-481c-aeb6-cac0b1a4f6bb service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:28:49.344555 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1152/2296] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:28:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:28:53.753305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c08eb6-eede-46b3-8313-65cae39abcf8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:53.753915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c08eb6-eede-46b3-8313-65cae39abcf8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:53.754389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c08eb6-eede-46b3-8313-65cae39abcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:53.781001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-46c08eb6-eede-46b3-8313-65cae39abcf8 None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:28:53.782636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-46c08eb6-eede-46b3-8313-65cae39abcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:53.790918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1145/2297] 10.222.0.64 () {58 vars in 967 bytes} [Wed Sep 21 16:28:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:28:53.842365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483074e6-5cca-481c-aeb6-cac0b1a4f6bb service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:28:53.855979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483074e6-5cca-481c-aeb6-cac0b1a4f6bb service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:28:53.861687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65857a35-1085-4c40-95c4-35c916fae8e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:28:53.862340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65857a35-1085-4c40-95c4-35c916fae8e0 service swift] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:53.862619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65857a35-1085-4c40-95c4-35c916fae8e0 service swift] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:53.862894 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65857a35-1085-4c40-95c4-35c916fae8e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:53.873513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65857a35-1085-4c40-95c4-35c916fae8e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:28:53.875578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65857a35-1085-4c40-95c4-35c916fae8e0 service swift] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:28:53.884118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1153/2298] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:28:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 21 16:28:54.343252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c08eb6-eede-46b3-8313-65cae39abcf8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:28:54.345744 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-46c08eb6-eede-46b3-8313-65cae39abcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:54.422839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c08eb6-eede-46b3-8313-65cae39abcf8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:28:54.428600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98c7a21-ec2d-46fd-a675-7d59b797ebab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:28:54.429818 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c7a21-ec2d-46fd-a675-7d59b797ebab service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:54.430337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c7a21-ec2d-46fd-a675-7d59b797ebab service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:54.430739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98c7a21-ec2d-46fd-a675-7d59b797ebab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:54.433480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c98c7a21-ec2d-46fd-a675-7d59b797ebab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:54.450310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c7a21-ec2d-46fd-a675-7d59b797ebab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:28:54.451937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98c7a21-ec2d-46fd-a675-7d59b797ebab service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:28:54.462385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1146/2299] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:28:54 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:28:57.850008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f25a0f-6d63-4d6c-861a-02bba3dcba88 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:57.850518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f25a0f-6d63-4d6c-861a-02bba3dcba88 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:57.850741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f25a0f-6d63-4d6c-861a-02bba3dcba88 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:57.852207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1154/2300] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:28:57.862730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06212d1e-96db-44a5-aa1f-41a5e861d155 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:57.863831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06212d1e-96db-44a5-aa1f-41a5e861d155 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:57.864035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06212d1e-96db-44a5-aa1f-41a5e861d155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:57.884206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-06212d1e-96db-44a5-aa1f-41a5e861d155 None None] MFA Rules not processed for user `55642294e8ff4f779510f4646e0e30a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:28:57.885412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-06212d1e-96db-44a5-aa1f-41a5e861d155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:57.893992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1147/2301] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:28:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:28:57.910838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f25a0f-6d63-4d6c-861a-02bba3dcba88 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:28:57.913311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a1f25a0f-6d63-4d6c-861a-02bba3dcba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:28:57.937027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f25a0f-6d63-4d6c-861a-02bba3dcba88 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:28:57.943605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee60de-6876-4965-bdd0-22a892e10ab5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:28:57.944254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee60de-6876-4965-bdd0-22a892e10ab5 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:28:57.944559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee60de-6876-4965-bdd0-22a892e10ab5 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:28:57.944799 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ee60de-6876-4965-bdd0-22a892e10ab5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:28:57.953910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee60de-6876-4965-bdd0-22a892e10ab5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:28:57.955610 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ee60de-6876-4965-bdd0-22a892e10ab5 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:28:57.967208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1155/2302] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:28:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:29:01.844135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06212d1e-96db-44a5-aa1f-41a5e861d155 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:01.846760 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-06212d1e-96db-44a5-aa1f-41a5e861d155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:01.930847 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06212d1e-96db-44a5-aa1f-41a5e861d155 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:01.936034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:01.936671 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:01.936956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:01.937388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] PATH_INFO: `/v3/users/70d318fe37c344649a165ef9a06a7b74` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:01.938592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d318fe37c344649a165ef9a06a7b74)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:01.940552 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:02.004538 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d318fe37c344649a165ef9a06a7b74'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.013075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d318fe37c344649a165ef9a06a7b74'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.030942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d318fe37c344649a165ef9a06a7b74'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.031714 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d318fe37c344649a165ef9a06a7b74'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.041423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:02.043886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d318fe37c344649a165ef9a06a7b74'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.052800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1148/2303] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:29:01 2022] DELETE /identity/v3/users/70d318fe37c344649a165ef9a06a7b74 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:02.256270 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee60de-6876-4965-bdd0-22a892e10ab5 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:02.267380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ee60de-6876-4965-bdd0-22a892e10ab5 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:02.330477 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:02.331037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:02.331320 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:02.331630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] PATH_INFO: `/v3/projects/f1939a4a2ffd40c49d04b42983fba762` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:02.333287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1939a4a2ffd40c49d04b42983fba762)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:02.335239 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:02.407225 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1939a4a2ffd40c49d04b42983fba762', 'request_id': 'req-c2e915c3-3d16-4482-8405-f16708a91640'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.416215 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1939a4a2ffd40c49d04b42983fba762', 'request_id': 'req-c2e915c3-3d16-4482-8405-f16708a91640'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.427557 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1939a4a2ffd40c49d04b42983fba762', 'request_id': 'req-c2e915c3-3d16-4482-8405-f16708a91640'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.439628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] The token cache is being invalidate because project f1939a4a2ffd40c49d04b42983fba762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:29:02.439919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.440601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.442684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1156/2304] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:29:02 2022] DELETE /identity/v3/projects/f1939a4a2ffd40c49d04b42983fba762 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:02.446939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:02.448398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:02.462277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3921d2-6550-4013-8553-ada442d065f9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:02.524074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebbf70-d06b-4f68-81ca-bd6d04679380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:02.524731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebbf70-d06b-4f68-81ca-bd6d04679380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:02.525039 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebbf70-d06b-4f68-81ca-bd6d04679380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:02.525316 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebbf70-d06b-4f68-81ca-bd6d04679380 admin admin] PATH_INFO: `/v3/users/3eea0f656490491e8fc151ebbb47769f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:02.526773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebbf70-d06b-4f68-81ca-bd6d04679380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eea0f656490491e8fc151ebbb47769f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:02.528828 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebbf70-d06b-4f68-81ca-bd6d04679380 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:02.585597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ebebbf70-d06b-4f68-81ca-bd6d04679380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eea0f656490491e8fc151ebbb47769f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.593027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ebebbf70-d06b-4f68-81ca-bd6d04679380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eea0f656490491e8fc151ebbb47769f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.606454 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ebebbf70-d06b-4f68-81ca-bd6d04679380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eea0f656490491e8fc151ebbb47769f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.607179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ebebbf70-d06b-4f68-81ca-bd6d04679380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eea0f656490491e8fc151ebbb47769f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.615157 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ebebbf70-d06b-4f68-81ca-bd6d04679380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eea0f656490491e8fc151ebbb47769f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:02.621548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1149/2305] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:29:02 2022] DELETE /identity/v3/users/3eea0f656490491e8fc151ebbb47769f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:02.840844 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:02.843198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:02.866321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e915c3-3d16-4482-8405-f16708a91640 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:02.919485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:02.955143 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:02.955797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:02.956081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:02.956353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] PATH_INFO: `/v3/projects/36e39fe8818a4b5697552d963c19f53b` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:02.957569 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36e39fe8818a4b5697552d963c19f53b)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:02.959544 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:03.030029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36e39fe8818a4b5697552d963c19f53b', 'request_id': 'req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:03.039440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36e39fe8818a4b5697552d963c19f53b', 'request_id': 'req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:03.047579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36e39fe8818a4b5697552d963c19f53b', 'request_id': 'req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:03.061148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] The token cache is being invalidate because project 36e39fe8818a4b5697552d963c19f53b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:29:03.061440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:03.062173 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:03.064632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1157/2306] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:29:02 2022] DELETE /identity/v3/projects/36e39fe8818a4b5697552d963c19f53b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:10.777217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699c8d0-8cda-4537-9390-1988babee4e1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:10.777803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699c8d0-8cda-4537-9390-1988babee4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:10.778001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699c8d0-8cda-4537-9390-1988babee4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:10.798471 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8699c8d0-8cda-4537-9390-1988babee4e1 None None] MFA Rules not processed for user `b917d84a650f4355a676e315ea039f0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:29:10.800212 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8699c8d0-8cda-4537-9390-1988babee4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:10.860020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1150/2307] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:29:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:29:11.194559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:11.196077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:11.256311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fff26d-fc57-4f56-a9e7-b1dbab8d4a00 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:11.260007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f322937-c6fe-483c-b6bd-a7058a147847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:11.260615 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f322937-c6fe-483c-b6bd-a7058a147847 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:11.260894 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f322937-c6fe-483c-b6bd-a7058a147847 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:11.261169 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f322937-c6fe-483c-b6bd-a7058a147847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:11.263089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4f322937-c6fe-483c-b6bd-a7058a147847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:11.274258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f322937-c6fe-483c-b6bd-a7058a147847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:11.275704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f322937-c6fe-483c-b6bd-a7058a147847 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:11.284447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1158/2308] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:29:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:29:19.182590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699c8d0-8cda-4537-9390-1988babee4e1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:19.184075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8699c8d0-8cda-4537-9390-1988babee4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:19.196446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699c8d0-8cda-4537-9390-1988babee4e1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:19.200055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a9eea-1552-4c2e-a483-929487e35073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:19.200660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a9eea-1552-4c2e-a483-929487e35073 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:19.200942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a9eea-1552-4c2e-a483-929487e35073 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:19.201218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a9eea-1552-4c2e-a483-929487e35073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:19.202973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-267a9eea-1552-4c2e-a483-929487e35073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:19.222646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:19.284950 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a9eea-1552-4c2e-a483-929487e35073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:19.286305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a9eea-1552-4c2e-a483-929487e35073 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:19.296235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1151/2309] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:29:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:29:21.280882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:22.592054 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f322937-c6fe-483c-b6bd-a7058a147847 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:22.593521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4f322937-c6fe-483c-b6bd-a7058a147847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:22.724197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f322937-c6fe-483c-b6bd-a7058a147847 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:22.729864 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f2148-abfd-4a13-ba5b-dc165463661a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:22.730678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f2148-abfd-4a13-ba5b-dc165463661a service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:22.731078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f2148-abfd-4a13-ba5b-dc165463661a service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:22.731348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f2148-abfd-4a13-ba5b-dc165463661a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:22.733520 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-781f2148-abfd-4a13-ba5b-dc165463661a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:22.748542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f2148-abfd-4a13-ba5b-dc165463661a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:22.751319 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f2148-abfd-4a13-ba5b-dc165463661a service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:22.769087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1159/2310] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:29:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:29:23.243166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a9eea-1552-4c2e-a483-929487e35073 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:23.244617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-267a9eea-1552-4c2e-a483-929487e35073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:23.259435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a9eea-1552-4c2e-a483-929487e35073 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:23.270007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb687d7-e02b-4a38-a972-d7cf4486c0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:23.270964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb687d7-e02b-4a38-a972-d7cf4486c0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:23.271733 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb687d7-e02b-4a38-a972-d7cf4486c0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:23.272012 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb687d7-e02b-4a38-a972-d7cf4486c0ec admin admin] PATH_INFO: `/v3/users/d2382a6610ae42fa865283ed12dd9982` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:23.273164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb687d7-e02b-4a38-a972-d7cf4486c0ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2382a6610ae42fa865283ed12dd9982)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:23.275104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb687d7-e02b-4a38-a972-d7cf4486c0ec admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:23.328564 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7fb687d7-e02b-4a38-a972-d7cf4486c0ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2382a6610ae42fa865283ed12dd9982'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:23.336803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7fb687d7-e02b-4a38-a972-d7cf4486c0ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2382a6610ae42fa865283ed12dd9982'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:23.350353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7fb687d7-e02b-4a38-a972-d7cf4486c0ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2382a6610ae42fa865283ed12dd9982'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:23.351027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7fb687d7-e02b-4a38-a972-d7cf4486c0ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2382a6610ae42fa865283ed12dd9982'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:23.358170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7fb687d7-e02b-4a38-a972-d7cf4486c0ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2382a6610ae42fa865283ed12dd9982'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:23.364986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1152/2311] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:29:23 2022] DELETE /identity/v3/users/d2382a6610ae42fa865283ed12dd9982 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:23.547509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f2148-abfd-4a13-ba5b-dc165463661a service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:23.562839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f2148-abfd-4a13-ba5b-dc165463661a service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:23.650584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83b16e-4250-456f-9b86-c92c905b95b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:23.651628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83b16e-4250-456f-9b86-c92c905b95b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:23.652064 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83b16e-4250-456f-9b86-c92c905b95b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:23.652481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83b16e-4250-456f-9b86-c92c905b95b3 admin admin] PATH_INFO: `/v3/projects/b93e202a295d427a85fdaf0209951bd5` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:23.653910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83b16e-4250-456f-9b86-c92c905b95b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b93e202a295d427a85fdaf0209951bd5)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:23.656062 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83b16e-4250-456f-9b86-c92c905b95b3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:23.740261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8c83b16e-4250-456f-9b86-c92c905b95b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b93e202a295d427a85fdaf0209951bd5', 'request_id': 'req-8c83b16e-4250-456f-9b86-c92c905b95b3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:23.748618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8c83b16e-4250-456f-9b86-c92c905b95b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b93e202a295d427a85fdaf0209951bd5', 'request_id': 'req-8c83b16e-4250-456f-9b86-c92c905b95b3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:23.756009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8c83b16e-4250-456f-9b86-c92c905b95b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b93e202a295d427a85fdaf0209951bd5', 'request_id': 'req-8c83b16e-4250-456f-9b86-c92c905b95b3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:23.766955 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8c83b16e-4250-456f-9b86-c92c905b95b3 admin admin] The token cache is being invalidate because project b93e202a295d427a85fdaf0209951bd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:29:23.767240 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8c83b16e-4250-456f-9b86-c92c905b95b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:23.767924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8c83b16e-4250-456f-9b86-c92c905b95b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:23.769738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1160/2312] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:29:23 2022] DELETE /identity/v3/projects/b93e202a295d427a85fdaf0209951bd5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:29:23.775293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cc619-b5fd-46fe-85bb-7ad9be17d37c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:23.775577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cc619-b5fd-46fe-85bb-7ad9be17d37c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:23.775853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cc619-b5fd-46fe-85bb-7ad9be17d37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:23.796394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4e1cc619-b5fd-46fe-85bb-7ad9be17d37c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:29:23.797695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4e1cc619-b5fd-46fe-85bb-7ad9be17d37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:23.860991 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1153/2313] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:29:23.869301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27d9fe-0003-499a-ae56-17368970e0c2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:23.869590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27d9fe-0003-499a-ae56-17368970e0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:23.869867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27d9fe-0003-499a-ae56-17368970e0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:23.889655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5b27d9fe-0003-499a-ae56-17368970e0c2 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:29:23.890896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5b27d9fe-0003-499a-ae56-17368970e0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:23.899560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1161/2314] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:29:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:29:23.904162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cc619-b5fd-46fe-85bb-7ad9be17d37c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:23.914395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cc619-b5fd-46fe-85bb-7ad9be17d37c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:23.919039 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cfeeb-bc6f-4d2f-a245-aaf384bffbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:23.919653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cfeeb-bc6f-4d2f-a245-aaf384bffbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:23.919932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cfeeb-bc6f-4d2f-a245-aaf384bffbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:23.920237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531cfeeb-bc6f-4d2f-a245-aaf384bffbb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:23.920884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cfeeb-bc6f-4d2f-a245-aaf384bffbb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:29:23.921188 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cfeeb-bc6f-4d2f-a245-aaf384bffbb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:23.922816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531cfeeb-bc6f-4d2f-a245-aaf384bffbb8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:23.935964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1154/2315] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:29:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:29:23.944494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27d9fe-0003-499a-ae56-17368970e0c2 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:23.956298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27d9fe-0003-499a-ae56-17368970e0c2 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:23.961280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f1790-4726-4366-80db-c39415598d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:23.961906 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f1790-4726-4366-80db-c39415598d41 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:23.962247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f1790-4726-4366-80db-c39415598d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:23.962521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f1790-4726-4366-80db-c39415598d41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:23.963157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f1790-4726-4366-80db-c39415598d41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:23.965070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55f1790-4726-4366-80db-c39415598d41 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:23.971890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:24.000741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1162/2316] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:29:23 2022] POST /identity/v3/projects => generated 403 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:29:24.005099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cfeeb-bc6f-4d2f-a245-aaf384bffbb8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:24.028929 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531cfeeb-bc6f-4d2f-a245-aaf384bffbb8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:24.039420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:24.091507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e9439-9253-4377-b706-5d31ecfd6acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:24.092153 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e9439-9253-4377-b706-5d31ecfd6acb admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:24.092415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e9439-9253-4377-b706-5d31ecfd6acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:24.093051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e9439-9253-4377-b706-5d31ecfd6acb admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:24.093361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e9439-9253-4377-b706-5d31ecfd6acb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:24.095320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e9439-9253-4377-b706-5d31ecfd6acb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:24.119370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1155/2317] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:29:24 2022] POST /identity/v3/users => generated 302 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:29:24.125339 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f1790-4726-4366-80db-c39415598d41 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:24.136867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f1790-4726-4366-80db-c39415598d41 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:24.141821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c6dab-c2ea-48ba-b74a-c9e9c7d123f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:24.143260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c6dab-c2ea-48ba-b74a-c9e9c7d123f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:24.143260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c6dab-c2ea-48ba-b74a-c9e9c7d123f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:24.143260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c6dab-c2ea-48ba-b74a-c9e9c7d123f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:24.143901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c6dab-c2ea-48ba-b74a-c9e9c7d123f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:24.145890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75c6dab-c2ea-48ba-b74a-c9e9c7d123f7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:24.158541 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1163/2318] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:29:24.163283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e9439-9253-4377-b706-5d31ecfd6acb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:24.182392 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e9439-9253-4377-b706-5d31ecfd6acb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:24.189745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf15c2-34f3-44e1-8016-30fd1255e281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:24.190731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf15c2-34f3-44e1-8016-30fd1255e281 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:24.191170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf15c2-34f3-44e1-8016-30fd1255e281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:24.191605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf15c2-34f3-44e1-8016-30fd1255e281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:24.192661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf15c2-34f3-44e1-8016-30fd1255e281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:24.195553 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf15c2-34f3-44e1-8016-30fd1255e281 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:24.213549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1156/2319] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:29:24 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:29:24.218194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c6dab-c2ea-48ba-b74a-c9e9c7d123f7 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:24.229856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c6dab-c2ea-48ba-b74a-c9e9c7d123f7 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:24.235171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bd2d5-4cd7-4e8e-b537-ed1d0d47a9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:24.236047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bd2d5-4cd7-4e8e-b537-ed1d0d47a9dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:24.236515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bd2d5-4cd7-4e8e-b537-ed1d0d47a9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:24.236894 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57bd2d5-4cd7-4e8e-b537-ed1d0d47a9dc admin admin] PATH_INFO: `/v3/projects/2c9cd6704a8d4524ac098874ab531bd9/users/8e213575c57445e7816aaddb5607edc4/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:24.260508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bd2d5-4cd7-4e8e-b537-ed1d0d47a9dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c9cd6704a8d4524ac098874ab531bd9, user_id=8e213575c57445e7816aaddb5607edc4, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:24.263316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57bd2d5-4cd7-4e8e-b537-ed1d0d47a9dc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:24.281004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1164/2320] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:29:24 2022] PUT /identity/v3/projects/2c9cd6704a8d4524ac098874ab531bd9/users/8e213575c57445e7816aaddb5607edc4/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:24.287106 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf3135-3b96-4a27-bf4f-b86549978ad8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:24.288001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf3135-3b96-4a27-bf4f-b86549978ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:24.288168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf3135-3b96-4a27-bf4f-b86549978ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:24.317295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8cdf3135-3b96-4a27-bf4f-b86549978ad8 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:29:24.319348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8cdf3135-3b96-4a27-bf4f-b86549978ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:24.418076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1157/2321] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:29:24.428113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bd2d5-4cd7-4e8e-b537-ed1d0d47a9dc admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:24.429944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b57bd2d5-4cd7-4e8e-b537-ed1d0d47a9dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:24.495865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57bd2d5-4cd7-4e8e-b537-ed1d0d47a9dc admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:24.499560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea07501-b6bb-45bb-a6d2-9237fb48783a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:24.500178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea07501-b6bb-45bb-a6d2-9237fb48783a service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:24.500458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea07501-b6bb-45bb-a6d2-9237fb48783a service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:24.500728 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea07501-b6bb-45bb-a6d2-9237fb48783a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:24.511618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea07501-b6bb-45bb-a6d2-9237fb48783a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:24.513212 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea07501-b6bb-45bb-a6d2-9237fb48783a service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:24.524510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1165/2322] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:29:25.389485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec442c7-a471-406b-a4c7-95ca44c05496 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:25.389975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec442c7-a471-406b-a4c7-95ca44c05496 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:25.390140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec442c7-a471-406b-a4c7-95ca44c05496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:25.427564 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8ec442c7-a471-406b-a4c7-95ca44c05496 None None] MFA Rules not processed for user `8e213575c57445e7816aaddb5607edc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:29:25.429795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8ec442c7-a471-406b-a4c7-95ca44c05496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:25.526396 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1158/2323] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:29:25 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:29:25.532155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea07501-b6bb-45bb-a6d2-9237fb48783a service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:25.544946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea07501-b6bb-45bb-a6d2-9237fb48783a service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:25.552545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c22d70-d519-4a56-8885-2f7f3d28c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:25.553735 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c22d70-d519-4a56-8885-2f7f3d28c3ad admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:25.555185 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c22d70-d519-4a56-8885-2f7f3d28c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:25.556278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c22d70-d519-4a56-8885-2f7f3d28c3ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:25.557324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c22d70-d519-4a56-8885-2f7f3d28c3ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:25.559795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c22d70-d519-4a56-8885-2f7f3d28c3ad admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:25.609649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1166/2324] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:29:25 2022] POST /identity/v3/projects => generated 403 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:29:25.614529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec442c7-a471-406b-a4c7-95ca44c05496 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:25.630190 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec442c7-a471-406b-a4c7-95ca44c05496 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:25.657382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:25.710364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310234b3-e021-40a4-94bc-f16b5522a64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:25.710984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310234b3-e021-40a4-94bc-f16b5522a64f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:25.711257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310234b3-e021-40a4-94bc-f16b5522a64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:25.711547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310234b3-e021-40a4-94bc-f16b5522a64f admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:25.712168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310234b3-e021-40a4-94bc-f16b5522a64f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:25.714115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310234b3-e021-40a4-94bc-f16b5522a64f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:25.737612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1159/2325] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:29:25 2022] POST /identity/v3/users => generated 308 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:29:25.741550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c22d70-d519-4a56-8885-2f7f3d28c3ad admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:25.753226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c22d70-d519-4a56-8885-2f7f3d28c3ad admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:25.758747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8690a10-9069-448a-8587-9f590430da98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:25.759633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8690a10-9069-448a-8587-9f590430da98 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:25.760092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8690a10-9069-448a-8587-9f590430da98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:25.761016 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8690a10-9069-448a-8587-9f590430da98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:25.761697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8690a10-9069-448a-8587-9f590430da98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:25.763358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8690a10-9069-448a-8587-9f590430da98 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:25.783673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1167/2326] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:29:25.787789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310234b3-e021-40a4-94bc-f16b5522a64f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:25.799793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310234b3-e021-40a4-94bc-f16b5522a64f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:25.804944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5d270-b307-4822-a0a8-c9f4201fe08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:25.805621 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5d270-b307-4822-a0a8-c9f4201fe08d admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:25.805906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5d270-b307-4822-a0a8-c9f4201fe08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:25.806222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5d270-b307-4822-a0a8-c9f4201fe08d admin admin] PATH_INFO: `/v3/projects/6741037c1ace40b9a6edfa31f461ce4a/users/ba006dcb193d4973bb2f80ed8f20f783/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:25.824466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5d270-b307-4822-a0a8-c9f4201fe08d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6741037c1ace40b9a6edfa31f461ce4a, user_id=ba006dcb193d4973bb2f80ed8f20f783, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:25.827339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5d270-b307-4822-a0a8-c9f4201fe08d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:25.842626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1160/2327] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:29:25 2022] PUT /identity/v3/projects/6741037c1ace40b9a6edfa31f461ce4a/users/ba006dcb193d4973bb2f80ed8f20f783/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:25.847530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8690a10-9069-448a-8587-9f590430da98 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:25.858309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8690a10-9069-448a-8587-9f590430da98 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:25.922515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02407e-595f-40aa-8a6a-dacf2154b312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:25.923075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02407e-595f-40aa-8a6a-dacf2154b312 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:25.923356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02407e-595f-40aa-8a6a-dacf2154b312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:25.923693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02407e-595f-40aa-8a6a-dacf2154b312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:25.924355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02407e-595f-40aa-8a6a-dacf2154b312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:25.926011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02407e-595f-40aa-8a6a-dacf2154b312 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:25.937756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1168/2328] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:29:25.942011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5d270-b307-4822-a0a8-c9f4201fe08d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:25.952830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5d270-b307-4822-a0a8-c9f4201fe08d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:25.958059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d42a1-7ed0-4f5a-9884-c26433c906b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:25.958737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d42a1-7ed0-4f5a-9884-c26433c906b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:25.959013 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d42a1-7ed0-4f5a-9884-c26433c906b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:25.959284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d42a1-7ed0-4f5a-9884-c26433c906b4 admin admin] PATH_INFO: `/v3/projects/6741037c1ace40b9a6edfa31f461ce4a/users/ba006dcb193d4973bb2f80ed8f20f783/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:25.961211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d42a1-7ed0-4f5a-9884-c26433c906b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6741037c1ace40b9a6edfa31f461ce4a, user_id=ba006dcb193d4973bb2f80ed8f20f783, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:25.964083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d42a1-7ed0-4f5a-9884-c26433c906b4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:25.975601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1161/2329] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:29:25 2022] PUT /identity/v3/projects/6741037c1ace40b9a6edfa31f461ce4a/users/ba006dcb193d4973bb2f80ed8f20f783/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:27.345632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d4fbc-ca29-4697-8d01-7a1bd2af601f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:27.346283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d4fbc-ca29-4697-8d01-7a1bd2af601f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:27.346476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d4fbc-ca29-4697-8d01-7a1bd2af601f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:27.409841 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:27.421680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-338d4fbc-ca29-4697-8d01-7a1bd2af601f None None] MFA Rules not processed for user `ba006dcb193d4973bb2f80ed8f20f783`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:29:27.423038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-338d4fbc-ca29-4697-8d01-7a1bd2af601f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:27.543792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1169/2330] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:29:27 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:29:28.931378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d42a1-7ed0-4f5a-9884-c26433c906b4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:28.933184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-615d42a1-7ed0-4f5a-9884-c26433c906b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:28.997116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d4fbc-ca29-4697-8d01-7a1bd2af601f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:29.059146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d42a1-7ed0-4f5a-9884-c26433c906b4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:29.065043 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd389a-d9e0-4ced-9f19-f2fe5623c533 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:29.065646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd389a-d9e0-4ced-9f19-f2fe5623c533 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:29.065935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd389a-d9e0-4ced-9f19-f2fe5623c533 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:29.066228 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd389a-d9e0-4ced-9f19-f2fe5623c533 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:29.067968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f9bd389a-d9e0-4ced-9f19-f2fe5623c533 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:29.080513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd389a-d9e0-4ced-9f19-f2fe5623c533 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:29.082023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd389a-d9e0-4ced-9f19-f2fe5623c533 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:29.083443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d4fbc-ca29-4697-8d01-7a1bd2af601f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:29.088415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:29.089126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:29.089479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:29.089821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] PATH_INFO: `/v3/users/8e213575c57445e7816aaddb5607edc4` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:29.091041 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e213575c57445e7816aaddb5607edc4)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:29.092025 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1162/2331] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:29:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:29:29.094855 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:29.145351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e213575c57445e7816aaddb5607edc4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.152956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e213575c57445e7816aaddb5607edc4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.153584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e213575c57445e7816aaddb5607edc4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.154861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:29.163141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e213575c57445e7816aaddb5607edc4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.175263 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e213575c57445e7816aaddb5607edc4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.181458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1170/2332] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:29:28 2022] DELETE /identity/v3/users/8e213575c57445e7816aaddb5607edc4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:29.367085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd389a-d9e0-4ced-9f19-f2fe5623c533 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:29.378856 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd389a-d9e0-4ced-9f19-f2fe5623c533 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:29.411086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:29.451843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:29.452485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:29.453043 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:29.453043 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] PATH_INFO: `/v3/projects/2c9cd6704a8d4524ac098874ab531bd9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:29.454196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c9cd6704a8d4524ac098874ab531bd9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:29.456624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:29.509811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c9cd6704a8d4524ac098874ab531bd9', 'request_id': 'req-57783f7c-af05-4d91-9313-6b71a93bbed1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.519346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c9cd6704a8d4524ac098874ab531bd9', 'request_id': 'req-57783f7c-af05-4d91-9313-6b71a93bbed1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.526196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c9cd6704a8d4524ac098874ab531bd9', 'request_id': 'req-57783f7c-af05-4d91-9313-6b71a93bbed1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.550879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] The token cache is being invalidate because project 2c9cd6704a8d4524ac098874ab531bd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:29:29.551168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.551813 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.553311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1163/2333] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:29:29 2022] DELETE /identity/v3/projects/2c9cd6704a8d4524ac098874ab531bd9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:29.557352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:29.558788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:29.572805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e104907-b8d6-4372-9468-10484821240b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:29.639029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8197e-fda5-46b8-9094-05da930d800d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:29.639679 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8197e-fda5-46b8-9094-05da930d800d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:29.639962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8197e-fda5-46b8-9094-05da930d800d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:29.640236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8197e-fda5-46b8-9094-05da930d800d admin admin] PATH_INFO: `/v3/users/ba006dcb193d4973bb2f80ed8f20f783` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:29.641382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8197e-fda5-46b8-9094-05da930d800d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba006dcb193d4973bb2f80ed8f20f783)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:29.643860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8197e-fda5-46b8-9094-05da930d800d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:29.714522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-24b8197e-fda5-46b8-9094-05da930d800d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba006dcb193d4973bb2f80ed8f20f783'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.725380 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-24b8197e-fda5-46b8-9094-05da930d800d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba006dcb193d4973bb2f80ed8f20f783'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.726503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-24b8197e-fda5-46b8-9094-05da930d800d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba006dcb193d4973bb2f80ed8f20f783'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.736291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-24b8197e-fda5-46b8-9094-05da930d800d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba006dcb193d4973bb2f80ed8f20f783'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.749892 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-24b8197e-fda5-46b8-9094-05da930d800d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba006dcb193d4973bb2f80ed8f20f783'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:29.756769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1171/2334] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:29:29 2022] DELETE /identity/v3/users/ba006dcb193d4973bb2f80ed8f20f783 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:29:29.941976 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:29.943641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:29.956492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57783f7c-af05-4d91-9313-6b71a93bbed1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:30.016139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:30.016778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.017117 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.017395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be admin admin] PATH_INFO: `/v3/projects/6741037c1ace40b9a6edfa31f461ce4a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.018590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be admin admin] RBAC: Authorizing `identity:delete_project(project_id=6741037c1ace40b9a6edfa31f461ce4a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:30.020744 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:30.066783 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:30.069373 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6741037c1ace40b9a6edfa31f461ce4a', 'request_id': 'req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:30.077489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6741037c1ace40b9a6edfa31f461ce4a', 'request_id': 'req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:30.083544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6741037c1ace40b9a6edfa31f461ce4a', 'request_id': 'req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:30.095981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be admin admin] The token cache is being invalidate because project 6741037c1ace40b9a6edfa31f461ce4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:29:30.096273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:30.096928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5c50db59-5cbf-4ea5-b8b5-df1476cc17be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:29:30.098451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1164/2335] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:29:29 2022] DELETE /identity/v3/projects/6741037c1ace40b9a6edfa31f461ce4a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:30.103407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96063e8-2507-49f4-961f-4e9723b36591 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.103996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96063e8-2507-49f4-961f-4e9723b36591 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.103996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96063e8-2507-49f4-961f-4e9723b36591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.167022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a96063e8-2507-49f4-961f-4e9723b36591 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:29:30.168255 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a96063e8-2507-49f4-961f-4e9723b36591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:30.182271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:30.230716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1172/2336] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:29:30.238642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd31a6-924c-457e-be43-0dbb18fb5cec None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.239000 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd31a6-924c-457e-be43-0dbb18fb5cec None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.239276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd31a6-924c-457e-be43-0dbb18fb5cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.259611 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b6cd31a6-924c-457e-be43-0dbb18fb5cec None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:29:30.260938 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b6cd31a6-924c-457e-be43-0dbb18fb5cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:30.270127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1165/2337] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:29:30.274501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96063e8-2507-49f4-961f-4e9723b36591 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:30.285058 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96063e8-2507-49f4-961f-4e9723b36591 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:30.289603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e50f97-ad90-4542-9829-0f14f4da28d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:30.290250 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e50f97-ad90-4542-9829-0f14f4da28d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.290528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e50f97-ad90-4542-9829-0f14f4da28d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.290802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e50f97-ad90-4542-9829-0f14f4da28d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.291450 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e50f97-ad90-4542-9829-0f14f4da28d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:29:30.291755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e50f97-ad90-4542-9829-0f14f4da28d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:30.293351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e50f97-ad90-4542-9829-0f14f4da28d1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:30.305768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1173/2338] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:29:30.309996 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd31a6-924c-457e-be43-0dbb18fb5cec None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:30.319300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd31a6-924c-457e-be43-0dbb18fb5cec None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:30.323979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c83d90-fde0-4e56-a07e-15056de60391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:30.324584 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c83d90-fde0-4e56-a07e-15056de60391 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.324860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c83d90-fde0-4e56-a07e-15056de60391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.325128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c83d90-fde0-4e56-a07e-15056de60391 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.325777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c83d90-fde0-4e56-a07e-15056de60391 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:30.327754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c83d90-fde0-4e56-a07e-15056de60391 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:30.371929 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1166/2339] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:29:30 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:29:30.376128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e50f97-ad90-4542-9829-0f14f4da28d1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:30.386607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e50f97-ad90-4542-9829-0f14f4da28d1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:30.445750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b07b1-28b0-44f5-8302-0c577e85c636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:30.446353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b07b1-28b0-44f5-8302-0c577e85c636 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.446666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b07b1-28b0-44f5-8302-0c577e85c636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.446935 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b07b1-28b0-44f5-8302-0c577e85c636 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.447613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b07b1-28b0-44f5-8302-0c577e85c636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:30.449538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b07b1-28b0-44f5-8302-0c577e85c636 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:30.474223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1174/2340] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:29:30 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:29:30.477493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c83d90-fde0-4e56-a07e-15056de60391 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:30.491164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c83d90-fde0-4e56-a07e-15056de60391 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:30.496701 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e1bf2-cf14-4c50-8346-821eff0ee9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:30.497314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e1bf2-cf14-4c50-8346-821eff0ee9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.497595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e1bf2-cf14-4c50-8346-821eff0ee9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.497868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e1bf2-cf14-4c50-8346-821eff0ee9d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.498604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e1bf2-cf14-4c50-8346-821eff0ee9d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:30.500132 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e1bf2-cf14-4c50-8346-821eff0ee9d2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:30.511198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1167/2341] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:29:30.515255 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b07b1-28b0-44f5-8302-0c577e85c636 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:30.526545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b07b1-28b0-44f5-8302-0c577e85c636 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:30.531324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070e2e7-34e8-4f63-9610-823c2a087572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:30.531937 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070e2e7-34e8-4f63-9610-823c2a087572 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.532248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070e2e7-34e8-4f63-9610-823c2a087572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.532588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070e2e7-34e8-4f63-9610-823c2a087572 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.533822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070e2e7-34e8-4f63-9610-823c2a087572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:30.535835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070e2e7-34e8-4f63-9610-823c2a087572 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:30.549026 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1175/2342] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:29:30.552973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e1bf2-cf14-4c50-8346-821eff0ee9d2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:30.564738 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e1bf2-cf14-4c50-8346-821eff0ee9d2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:30.569763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36180c-6a7e-42d1-b85d-95edf1ea4902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:30.570559 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36180c-6a7e-42d1-b85d-95edf1ea4902 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.570839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36180c-6a7e-42d1-b85d-95edf1ea4902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.571110 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36180c-6a7e-42d1-b85d-95edf1ea4902 admin admin] PATH_INFO: `/v3/projects/f7157ce8a3b347fc8912677b9316640e/users/5fe5cc1c4fe4443488e5a51c79033fbf/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.596075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36180c-6a7e-42d1-b85d-95edf1ea4902 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7157ce8a3b347fc8912677b9316640e, user_id=5fe5cc1c4fe4443488e5a51c79033fbf, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:30.599012 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca36180c-6a7e-42d1-b85d-95edf1ea4902 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:30.615527 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1168/2343] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:29:30 2022] PUT /identity/v3/projects/f7157ce8a3b347fc8912677b9316640e/users/5fe5cc1c4fe4443488e5a51c79033fbf/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:30.628567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c8859-cce4-40c6-a854-ebce3cefa1e4 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.629053 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c8859-cce4-40c6-a854-ebce3cefa1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.629468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c8859-cce4-40c6-a854-ebce3cefa1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.662708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c22c8859-cce4-40c6-a854-ebce3cefa1e4 None None] MFA Rules not processed for user `5fe5cc1c4fe4443488e5a51c79033fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:29:30.663944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c22c8859-cce4-40c6-a854-ebce3cefa1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:30.733579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1176/2344] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:29:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:29:30.738257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36180c-6a7e-42d1-b85d-95edf1ea4902 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:30.802592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36180c-6a7e-42d1-b85d-95edf1ea4902 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:30.807278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb808261-3d83-4ecb-8260-691a31c09781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:30.807884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb808261-3d83-4ecb-8260-691a31c09781 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.808162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb808261-3d83-4ecb-8260-691a31c09781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.808433 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb808261-3d83-4ecb-8260-691a31c09781 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.809062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb808261-3d83-4ecb-8260-691a31c09781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:30.817488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb808261-3d83-4ecb-8260-691a31c09781 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:30.840835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1169/2345] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:29:30 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:29:30.845910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c8859-cce4-40c6-a854-ebce3cefa1e4 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:30.857060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c8859-cce4-40c6-a854-ebce3cefa1e4 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:30.871176 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:30.922736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783fb97-4945-4e4a-895f-f3af124b9777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:30.924050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783fb97-4945-4e4a-895f-f3af124b9777 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.924332 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783fb97-4945-4e4a-895f-f3af124b9777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.924605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783fb97-4945-4e4a-895f-f3af124b9777 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.925278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783fb97-4945-4e4a-895f-f3af124b9777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:30.927277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783fb97-4945-4e4a-895f-f3af124b9777 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:30.957777 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1177/2346] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:29:30 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:29:30.961678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb808261-3d83-4ecb-8260-691a31c09781 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:30.973908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb808261-3d83-4ecb-8260-691a31c09781 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:30.978737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb55b72-c27a-4eb8-a7c8-b52b7fa0ef7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:30.979358 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb55b72-c27a-4eb8-a7c8-b52b7fa0ef7a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:30.979636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb55b72-c27a-4eb8-a7c8-b52b7fa0ef7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:30.979901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb55b72-c27a-4eb8-a7c8-b52b7fa0ef7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:30.980703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb55b72-c27a-4eb8-a7c8-b52b7fa0ef7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:30.982343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb55b72-c27a-4eb8-a7c8-b52b7fa0ef7a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:30.994512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1170/2347] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:29:31.000283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783fb97-4945-4e4a-895f-f3af124b9777 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:31.011019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783fb97-4945-4e4a-895f-f3af124b9777 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:31.015995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5ee4d-d1d3-42c4-8a1c-96a40a4732ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:31.016673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5ee4d-d1d3-42c4-8a1c-96a40a4732ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:31.017567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5ee4d-d1d3-42c4-8a1c-96a40a4732ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:31.017838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd5ee4d-d1d3-42c4-8a1c-96a40a4732ba admin admin] PATH_INFO: `/v3/projects/905198a4c0304fcbab3cf0989c2c3233/users/bd6c27a24d754bdd834c36f608791529/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:31.036191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5ee4d-d1d3-42c4-8a1c-96a40a4732ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=905198a4c0304fcbab3cf0989c2c3233, user_id=bd6c27a24d754bdd834c36f608791529, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:31.039032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd5ee4d-d1d3-42c4-8a1c-96a40a4732ba admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:31.054424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1178/2348] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:29:30 2022] PUT /identity/v3/projects/905198a4c0304fcbab3cf0989c2c3233/users/bd6c27a24d754bdd834c36f608791529/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:29:31.059244 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb55b72-c27a-4eb8-a7c8-b52b7fa0ef7a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:31.073687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb55b72-c27a-4eb8-a7c8-b52b7fa0ef7a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:31.117346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:31.147344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db75ff3-5e51-42f7-be53-cb6b08cfc2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:31.147962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db75ff3-5e51-42f7-be53-cb6b08cfc2bf admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:31.148443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db75ff3-5e51-42f7-be53-cb6b08cfc2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:31.148632 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db75ff3-5e51-42f7-be53-cb6b08cfc2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:31.149600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db75ff3-5e51-42f7-be53-cb6b08cfc2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:31.150796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db75ff3-5e51-42f7-be53-cb6b08cfc2bf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:31.163735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1171/2349] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:29:31.167989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5ee4d-d1d3-42c4-8a1c-96a40a4732ba admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:31.180309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd5ee4d-d1d3-42c4-8a1c-96a40a4732ba admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:31.184664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6244f01-ace9-48d8-bc4a-71e4facf7846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:31.185375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6244f01-ace9-48d8-bc4a-71e4facf7846 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:31.185642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6244f01-ace9-48d8-bc4a-71e4facf7846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:31.185914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6244f01-ace9-48d8-bc4a-71e4facf7846 admin admin] PATH_INFO: `/v3/projects/905198a4c0304fcbab3cf0989c2c3233/users/bd6c27a24d754bdd834c36f608791529/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:31.187950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6244f01-ace9-48d8-bc4a-71e4facf7846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=905198a4c0304fcbab3cf0989c2c3233, user_id=bd6c27a24d754bdd834c36f608791529, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:31.190682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6244f01-ace9-48d8-bc4a-71e4facf7846 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:31.201607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1179/2350] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:29:31 2022] PUT /identity/v3/projects/905198a4c0304fcbab3cf0989c2c3233/users/bd6c27a24d754bdd834c36f608791529/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:29:31.212735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df452e03-9bb3-495a-94d0-cb4182ac8b66 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:31.213016 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df452e03-9bb3-495a-94d0-cb4182ac8b66 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:31.213287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df452e03-9bb3-495a-94d0-cb4182ac8b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:31.245440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-df452e03-9bb3-495a-94d0-cb4182ac8b66 None None] MFA Rules not processed for user `bd6c27a24d754bdd834c36f608791529`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:29:31.246630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-df452e03-9bb3-495a-94d0-cb4182ac8b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:31.322721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1172/2351] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:29:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:29:31.334280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6244f01-ace9-48d8-bc4a-71e4facf7846 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:31.336055 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a6244f01-ace9-48d8-bc4a-71e4facf7846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:31.434705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6244f01-ace9-48d8-bc4a-71e4facf7846 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:31.438511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98bd19-0ba9-43d2-979c-91257dd0ee75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:31.439159 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98bd19-0ba9-43d2-979c-91257dd0ee75 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:31.439435 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98bd19-0ba9-43d2-979c-91257dd0ee75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:31.439703 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98bd19-0ba9-43d2-979c-91257dd0ee75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:31.508024 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98bd19-0ba9-43d2-979c-91257dd0ee75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:31.509578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98bd19-0ba9-43d2-979c-91257dd0ee75 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:31.519793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1180/2352] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:29:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:29:37.334386 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df452e03-9bb3-495a-94d0-cb4182ac8b66 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:37.345075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df452e03-9bb3-495a-94d0-cb4182ac8b66 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:37.348810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51bc71-c926-4914-9254-63270cc46bfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:37.349423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51bc71-c926-4914-9254-63270cc46bfe service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:37.349701 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51bc71-c926-4914-9254-63270cc46bfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:37.349972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51bc71-c926-4914-9254-63270cc46bfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:37.359869 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51bc71-c926-4914-9254-63270cc46bfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:37.361946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51bc71-c926-4914-9254-63270cc46bfe service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:37.372529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1173/2353] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:29:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:29:41.479853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb07a0-8ae5-48ba-baec-d8ac94fd4a44 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:41.480451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb07a0-8ae5-48ba-baec-d8ac94fd4a44 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:41.480451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb07a0-8ae5-48ba-baec-d8ac94fd4a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:41.488716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:41.532523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-3fdb07a0-8ae5-48ba-baec-d8ac94fd4a44 None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:29:41.534651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3fdb07a0-8ae5-48ba-baec-d8ac94fd4a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:41.630648 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1181/2354] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:29:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:29:41.649252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51bc71-c926-4914-9254-63270cc46bfe service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:41.650766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bf51bc71-c926-4914-9254-63270cc46bfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:41.737804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51bc71-c926-4914-9254-63270cc46bfe service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:41.745539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b61cb-b89f-4fd9-a1e2-a8b6e043e01c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:41.746293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b61cb-b89f-4fd9-a1e2-a8b6e043e01c service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:41.746573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b61cb-b89f-4fd9-a1e2-a8b6e043e01c service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:41.746852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b61cb-b89f-4fd9-a1e2-a8b6e043e01c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:41.759340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b61cb-b89f-4fd9-a1e2-a8b6e043e01c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:41.760982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b61cb-b89f-4fd9-a1e2-a8b6e043e01c service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:41.771299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1174/2355] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:29:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 16:29:48.020217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c17c2d-7aab-45fe-8146-792603d453a0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:48.021103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c17c2d-7aab-45fe-8146-792603d453a0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:48.021103 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c17c2d-7aab-45fe-8146-792603d453a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:48.075240 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e1c17c2d-7aab-45fe-8146-792603d453a0 None None] MFA Rules not processed for user `f619261c21204ea3b5ba624177eb8747`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:29:48.077985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e1c17c2d-7aab-45fe-8146-792603d453a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:48.174434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1182/2356] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:29:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:29:51.774203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=85 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:58.160084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=53 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:29:59.759584 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b61cb-b89f-4fd9-a1e2-a8b6e043e01c service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:29:59.761481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d16b61cb-b89f-4fd9-a1e2-a8b6e043e01c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:59.866089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b61cb-b89f-4fd9-a1e2-a8b6e043e01c service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:29:59.872449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24d55c-f5ac-4183-8f9c-28e1233a3565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:29:59.873367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24d55c-f5ac-4183-8f9c-28e1233a3565 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:29:59.873810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24d55c-f5ac-4183-8f9c-28e1233a3565 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:29:59.874878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24d55c-f5ac-4183-8f9c-28e1233a3565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:29:59.877154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3e24d55c-f5ac-4183-8f9c-28e1233a3565 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:29:59.979166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24d55c-f5ac-4183-8f9c-28e1233a3565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:29:59.980754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24d55c-f5ac-4183-8f9c-28e1233a3565 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:29:59.994925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1175/2357] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:29:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:00.784431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e26e5a-00fc-4d91-9b3c-968e12b51d51 None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:00.786344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e26e5a-00fc-4d91-9b3c-968e12b51d51 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:00.786785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e26e5a-00fc-4d91-9b3c-968e12b51d51 None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:00.790656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1183/2358] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:30:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:30:00.802511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7644ddb-4fa1-4d1b-bb97-3dccd89d7dbf None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:00.803161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7644ddb-4fa1-4d1b-bb97-3dccd89d7dbf None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:00.805507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7644ddb-4fa1-4d1b-bb97-3dccd89d7dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:00.830459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b7644ddb-4fa1-4d1b-bb97-3dccd89d7dbf None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:00.831947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b7644ddb-4fa1-4d1b-bb97-3dccd89d7dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:00.914955 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1176/2359] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:30:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:00.927887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e26e5a-00fc-4d91-9b3c-968e12b51d51 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:00.929766 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a1e26e5a-00fc-4d91-9b3c-968e12b51d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:01.000524 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e26e5a-00fc-4d91-9b3c-968e12b51d51 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:01.005128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b97939-3b44-4d41-a261-5026ab0625a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2d76fb9cbdd40f9b2a748209cee5b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:01.005796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b97939-3b44-4d41-a261-5026ab0625a2 service glance] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:01.006084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b97939-3b44-4d41-a261-5026ab0625a2 service glance] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:01.006357 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b97939-3b44-4d41-a261-5026ab0625a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:01.019262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b97939-3b44-4d41-a261-5026ab0625a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:01.020707 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b97939-3b44-4d41-a261-5026ab0625a2 service glance] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:01.039205 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1184/2360] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:01.132754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711848ca-6b68-4518-9a36-77309b1a1e95 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:01.133265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711848ca-6b68-4518-9a36-77309b1a1e95 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:01.133677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711848ca-6b68-4518-9a36-77309b1a1e95 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:01.135547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1177/2361] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:30:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:30:01.142078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43207621-f465-461b-a36d-550386bda34f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:01.142429 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43207621-f465-461b-a36d-550386bda34f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:01.143113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43207621-f465-461b-a36d-550386bda34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:01.167673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-43207621-f465-461b-a36d-550386bda34f None None] MFA Rules not processed for user `e175b8de037d45d48cbddd64e552f6f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:01.168587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-43207621-f465-461b-a36d-550386bda34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:01.178388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1185/2362] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:30:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:01.193745 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711848ca-6b68-4518-9a36-77309b1a1e95 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:01.195767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-711848ca-6b68-4518-9a36-77309b1a1e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:01.217175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711848ca-6b68-4518-9a36-77309b1a1e95 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:01.225412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabb075e-7f36-4dd1-9df1-270a85dd6c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:01.226604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb075e-7f36-4dd1-9df1-270a85dd6c93 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:01.227101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb075e-7f36-4dd1-9df1-270a85dd6c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:01.227568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabb075e-7f36-4dd1-9df1-270a85dd6c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:01.242689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb075e-7f36-4dd1-9df1-270a85dd6c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:01.245564 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabb075e-7f36-4dd1-9df1-270a85dd6c93 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:01.263435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1178/2363] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:01.377740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094c407-d355-4b0e-a252-d5cf441f0daa None None] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:01.378030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094c407-d355-4b0e-a252-d5cf441f0daa None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:01.378346 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094c407-d355-4b0e-a252-d5cf441f0daa None None] PATH_INFO: `/` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:01.379924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1186/2364] 10.222.0.64 () {54 vars in 921 bytes} [Wed Sep 21 16:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 21 16:30:01.386472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b7bc9-e33a-4766-b4eb-e14d2543072a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:01.386784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b7bc9-e33a-4766-b4eb-e14d2543072a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:01.387170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b7bc9-e33a-4766-b4eb-e14d2543072a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:01.420717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8b1b7bc9-e33a-4766-b4eb-e14d2543072a None None] MFA Rules not processed for user `f0acf23d224e4b16ad03cdd9bf7cae8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:01.423948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8b1b7bc9-e33a-4766-b4eb-e14d2543072a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:01.438445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1179/2365] 10.222.0.64 () {58 vars in 1018 bytes} [Wed Sep 21 16:30:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:30:01.455806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094c407-d355-4b0e-a252-d5cf441f0daa None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:01.473434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094c407-d355-4b0e-a252-d5cf441f0daa None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:01.478535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a5f6fd-1509-4c0c-b070-a3ae6c1dc8f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:01.480838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5f6fd-1509-4c0c-b070-a3ae6c1dc8f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:01.480838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5f6fd-1509-4c0c-b070-a3ae6c1dc8f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:01.480838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a5f6fd-1509-4c0c-b070-a3ae6c1dc8f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:01.500294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5f6fd-1509-4c0c-b070-a3ae6c1dc8f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:01.501912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a5f6fd-1509-4c0c-b070-a3ae6c1dc8f8 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:01.520744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1187/2366] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:02.849498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b7bc9-e33a-4766-b4eb-e14d2543072a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:02.857520 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:02.865278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b7bc9-e33a-4766-b4eb-e14d2543072a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:02.876484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:02.877704 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:02.878304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:02.878818 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] PATH_INFO: `/v3/users/5fe5cc1c4fe4443488e5a51c79033fbf` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:02.880761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fe5cc1c4fe4443488e5a51c79033fbf)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:02.883215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:02.959452 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fe5cc1c4fe4443488e5a51c79033fbf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:02.971970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fe5cc1c4fe4443488e5a51c79033fbf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:03.005468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fe5cc1c4fe4443488e5a51c79033fbf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:03.006728 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fe5cc1c4fe4443488e5a51c79033fbf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:03.049558 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fe5cc1c4fe4443488e5a51c79033fbf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:03.070037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1180/2367] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:02 2022] DELETE /identity/v3/users/5fe5cc1c4fe4443488e5a51c79033fbf => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:03.071357 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bf497-de94-4ff2-86ea-4a586846cb98 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:03.072063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bf497-de94-4ff2-86ea-4a586846cb98 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:03.072063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bf497-de94-4ff2-86ea-4a586846cb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:03.120184 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-412bf497-de94-4ff2-86ea-4a586846cb98 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:03.121770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-412bf497-de94-4ff2-86ea-4a586846cb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:03.288162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1188/2368] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:30:03.325910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:03.328577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:03.428451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe49fa-b0a5-4da9-809c-108f36cf45e9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:03.432807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38266933-f250-4059-9a9d-a6a9556ca033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:03.433549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38266933-f250-4059-9a9d-a6a9556ca033 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:03.433928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38266933-f250-4059-9a9d-a6a9556ca033 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:03.434310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38266933-f250-4059-9a9d-a6a9556ca033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:03.446114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38266933-f250-4059-9a9d-a6a9556ca033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:03.447596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38266933-f250-4059-9a9d-a6a9556ca033 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:03.458517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1181/2369] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:04.185919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bf497-de94-4ff2-86ea-4a586846cb98 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:04.187653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-412bf497-de94-4ff2-86ea-4a586846cb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:04.211774 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bf497-de94-4ff2-86ea-4a586846cb98 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:04.224139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:04.224812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:04.226340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:04.226631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] PATH_INFO: `/v3/projects/f7157ce8a3b347fc8912677b9316640e` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:04.228920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7157ce8a3b347fc8912677b9316640e)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:04.232277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:04.367711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7157ce8a3b347fc8912677b9316640e', 'request_id': 'req-a8f136ee-fae5-488e-855e-594a7318057f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:04.395918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7157ce8a3b347fc8912677b9316640e', 'request_id': 'req-a8f136ee-fae5-488e-855e-594a7318057f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:04.398339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece8bab-6011-42b4-b559-7613ec24c92e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:04.398827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece8bab-6011-42b4-b559-7613ec24c92e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:04.398910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece8bab-6011-42b4-b559-7613ec24c92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:04.410244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7157ce8a3b347fc8912677b9316640e', 'request_id': 'req-a8f136ee-fae5-488e-855e-594a7318057f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:04.434893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3ece8bab-6011-42b4-b559-7613ec24c92e None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:04.436089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] The token cache is being invalidate because project f7157ce8a3b347fc8912677b9316640e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:04.436364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3ece8bab-6011-42b4-b559-7613ec24c92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:04.436485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:04.437556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:04.439934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1189/2370] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:04 2022] DELETE /identity/v3/projects/f7157ce8a3b347fc8912677b9316640e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:04.445335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:04.447030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:04.462782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:04.520502 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1182/2371] 10.222.0.64 () {58 vars in 1009 bytes} [Wed Sep 21 16:30:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:30:04.544060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f136ee-fae5-488e-855e-594a7318057f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:04.549780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:04.550591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:04.550874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:04.551225 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] PATH_INFO: `/v3/users/bd6c27a24d754bdd834c36f608791529` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:04.551629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece8bab-6011-42b4-b559-7613ec24c92e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:04.552466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd6c27a24d754bdd834c36f608791529)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:04.553098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3ece8bab-6011-42b4-b559-7613ec24c92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:04.554561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:04.588468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:04.616824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd6c27a24d754bdd834c36f608791529'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:04.631790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd6c27a24d754bdd834c36f608791529'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:04.631790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd6c27a24d754bdd834c36f608791529'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:04.650915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd6c27a24d754bdd834c36f608791529'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:04.672859 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece8bab-6011-42b4-b559-7613ec24c92e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:04.680693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dc1c5-554a-49ec-8d9d-3a8536fbc10c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:04.681379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dc1c5-554a-49ec-8d9d-3a8536fbc10c service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:04.681670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dc1c5-554a-49ec-8d9d-3a8536fbc10c service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:04.681952 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dc1c5-554a-49ec-8d9d-3a8536fbc10c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:04.682410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd6c27a24d754bdd834c36f608791529'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:04.683953 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-471dc1c5-554a-49ec-8d9d-3a8536fbc10c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:04.698142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1190/2372] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:04 2022] DELETE /identity/v3/users/bd6c27a24d754bdd834c36f608791529 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:04.703989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471dc1c5-554a-49ec-8d9d-3a8536fbc10c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:04.705827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471dc1c5-554a-49ec-8d9d-3a8536fbc10c service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:04.724566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1183/2373] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:30:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 16:30:05.018740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:05.022137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:05.265098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cc6da-22a6-402e-831d-2bb370e27bc8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:05.271724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f257364-5e65-4bc6-8133-862679cde429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:05.274353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f257364-5e65-4bc6-8133-862679cde429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:05.275369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f257364-5e65-4bc6-8133-862679cde429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:05.275837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f257364-5e65-4bc6-8133-862679cde429 admin admin] PATH_INFO: `/v3/users/2e5204475ce7477f9cfeeffed457a445` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:05.279425 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f257364-5e65-4bc6-8133-862679cde429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5204475ce7477f9cfeeffed457a445)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:05.285611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f257364-5e65-4bc6-8133-862679cde429 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:05.352938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:05.389468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7f257364-5e65-4bc6-8133-862679cde429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5204475ce7477f9cfeeffed457a445'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:05.413148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7f257364-5e65-4bc6-8133-862679cde429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5204475ce7477f9cfeeffed457a445'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:05.414468 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7f257364-5e65-4bc6-8133-862679cde429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5204475ce7477f9cfeeffed457a445'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:05.445006 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7f257364-5e65-4bc6-8133-862679cde429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5204475ce7477f9cfeeffed457a445'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:05.466935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dc1c5-554a-49ec-8d9d-3a8536fbc10c service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:05.469734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-471dc1c5-554a-49ec-8d9d-3a8536fbc10c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:05.474040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7f257364-5e65-4bc6-8133-862679cde429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5204475ce7477f9cfeeffed457a445'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:05.486689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1191/2374] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:05 2022] DELETE /identity/v3/users/2e5204475ce7477f9cfeeffed457a445 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:05.493366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dc1c5-554a-49ec-8d9d-3a8536fbc10c service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:05.667721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:05.671226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:05.672064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:05.673287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] PATH_INFO: `/v3/projects/905198a4c0304fcbab3cf0989c2c3233` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:05.677087 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=905198a4c0304fcbab3cf0989c2c3233)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:05.681388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:05.776012 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '905198a4c0304fcbab3cf0989c2c3233', 'request_id': 'req-afe4275e-1106-4949-8a57-efec14093c12'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:05.792910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '905198a4c0304fcbab3cf0989c2c3233', 'request_id': 'req-afe4275e-1106-4949-8a57-efec14093c12'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:05.803677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '905198a4c0304fcbab3cf0989c2c3233', 'request_id': 'req-afe4275e-1106-4949-8a57-efec14093c12'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:05.829392 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] The token cache is being invalidate because project 905198a4c0304fcbab3cf0989c2c3233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:05.829917 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:05.830409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:05.835641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1184/2375] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:05 2022] DELETE /identity/v3/projects/905198a4c0304fcbab3cf0989c2c3233 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:05.841128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c67ca-7e85-4d04-aa8e-6d8dc04e4162 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:05.841459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c67ca-7e85-4d04-aa8e-6d8dc04e4162 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:05.841950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c67ca-7e85-4d04-aa8e-6d8dc04e4162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:05.844821 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:05.846424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:05.882580 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-254c67ca-7e85-4d04-aa8e-6d8dc04e4162 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:05.883987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-254c67ca-7e85-4d04-aa8e-6d8dc04e4162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:05.943420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:05.991897 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe4275e-1106-4949-8a57-efec14093c12 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:05.996788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888b645-3bd8-443b-92ce-338f14534aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:05.997330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888b645-3bd8-443b-92ce-338f14534aca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:05.997764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888b645-3bd8-443b-92ce-338f14534aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:05.998027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888b645-3bd8-443b-92ce-338f14534aca admin admin] PATH_INFO: `/v3/projects/9c326adfa14149e5ae1d4eb45f4807b8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:05.999394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888b645-3bd8-443b-92ce-338f14534aca admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c326adfa14149e5ae1d4eb45f4807b8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:06.001089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1192/2376] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:06.001940 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888b645-3bd8-443b-92ce-338f14534aca admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:06.016681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901d57e-a4f5-434d-9933-f86c5d7b1a93 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.016964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901d57e-a4f5-434d-9933-f86c5d7b1a93 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.017545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901d57e-a4f5-434d-9933-f86c5d7b1a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.052910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d901d57e-a4f5-434d-9933-f86c5d7b1a93 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:06.054726 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d901d57e-a4f5-434d-9933-f86c5d7b1a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:06.072968 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1193/2377] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:06.074985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901d57e-a4f5-434d-9933-f86c5d7b1a93 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:06.089123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901d57e-a4f5-434d-9933-f86c5d7b1a93 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:06.104587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0888b645-3bd8-443b-92ce-338f14534aca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c326adfa14149e5ae1d4eb45f4807b8', 'request_id': 'req-0888b645-3bd8-443b-92ce-338f14534aca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:06.121511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0888b645-3bd8-443b-92ce-338f14534aca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c326adfa14149e5ae1d4eb45f4807b8', 'request_id': 'req-0888b645-3bd8-443b-92ce-338f14534aca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:06.132030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0888b645-3bd8-443b-92ce-338f14534aca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c326adfa14149e5ae1d4eb45f4807b8', 'request_id': 'req-0888b645-3bd8-443b-92ce-338f14534aca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:06.163462 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0888b645-3bd8-443b-92ce-338f14534aca admin admin] The token cache is being invalidate because project 9c326adfa14149e5ae1d4eb45f4807b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:06.163928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0888b645-3bd8-443b-92ce-338f14534aca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:06.164751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0888b645-3bd8-443b-92ce-338f14534aca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:06.167712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1185/2378] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:05 2022] DELETE /identity/v3/projects/9c326adfa14149e5ae1d4eb45f4807b8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:06.176992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f98b50-73d8-4c68-9734-516b243807c9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.177493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f98b50-73d8-4c68-9734-516b243807c9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.178234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f98b50-73d8-4c68-9734-516b243807c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.197125 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c48cd5-5e51-428d-8e4b-44f634cb11df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:06.199979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c48cd5-5e51-428d-8e4b-44f634cb11df admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.200638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c48cd5-5e51-428d-8e4b-44f634cb11df admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.201031 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c48cd5-5e51-428d-8e4b-44f634cb11df admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.207420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c48cd5-5e51-428d-8e4b-44f634cb11df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:06.207964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c48cd5-5e51-428d-8e4b-44f634cb11df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:06.212213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c48cd5-5e51-428d-8e4b-44f634cb11df admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:06.227031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d2f98b50-73d8-4c68-9734-516b243807c9 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:06.228276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d2f98b50-73d8-4c68-9734-516b243807c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:06.237447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1194/2379] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:30:06.244834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c48cd5-5e51-428d-8e4b-44f634cb11df admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:06.245024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1186/2380] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:06.246307 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a4c48cd5-5e51-428d-8e4b-44f634cb11df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:06.246877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340160dd-3a46-44eb-8fa1-dd61118a9e98 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.247177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340160dd-3a46-44eb-8fa1-dd61118a9e98 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.247870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340160dd-3a46-44eb-8fa1-dd61118a9e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.262982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c48cd5-5e51-428d-8e4b-44f634cb11df admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:06.269550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58218c1-9ee8-4c8e-9aab-f23566466b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:06.270370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58218c1-9ee8-4c8e-9aab-f23566466b0b admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.271027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58218c1-9ee8-4c8e-9aab-f23566466b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.271210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58218c1-9ee8-4c8e-9aab-f23566466b0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.271908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58218c1-9ee8-4c8e-9aab-f23566466b0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:06.273064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-340160dd-3a46-44eb-8fa1-dd61118a9e98 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:06.273939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58218c1-9ee8-4c8e-9aab-f23566466b0b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:06.274388 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-340160dd-3a46-44eb-8fa1-dd61118a9e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:06.288267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1187/2381] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:30:06.289718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340160dd-3a46-44eb-8fa1-dd61118a9e98 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:06.303988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340160dd-3a46-44eb-8fa1-dd61118a9e98 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:06.309567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8f63e-ecf6-412a-b0d0-a8edafa9441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:06.310476 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8f63e-ecf6-412a-b0d0-a8edafa9441a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.310755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8f63e-ecf6-412a-b0d0-a8edafa9441a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.311080 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8f63e-ecf6-412a-b0d0-a8edafa9441a admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.312254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8f63e-ecf6-412a-b0d0-a8edafa9441a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:06.312767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8f63e-ecf6-412a-b0d0-a8edafa9441a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:06.312894 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1195/2382] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:06 2022] POST /identity/v3/projects => generated 379 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:06.314349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8f63e-ecf6-412a-b0d0-a8edafa9441a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:06.317428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58218c1-9ee8-4c8e-9aab-f23566466b0b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:06.347816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1188/2383] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:30:06.349146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8f63e-ecf6-412a-b0d0-a8edafa9441a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:06.361243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8f63e-ecf6-412a-b0d0-a8edafa9441a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:06.363836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:06.453657 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58218c1-9ee8-4c8e-9aab-f23566466b0b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:06.461314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b1d0b-230c-424c-95f9-34f4fd6da6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:06.462083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b1d0b-230c-424c-95f9-34f4fd6da6c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.462428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b1d0b-230c-424c-95f9-34f4fd6da6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.462641 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b1d0b-230c-424c-95f9-34f4fd6da6c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.463458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b1d0b-230c-424c-95f9-34f4fd6da6c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:06.468086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b1d0b-230c-424c-95f9-34f4fd6da6c4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:06.502453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368098a-b4fc-4d83-963b-d4835e7dc8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:06.504341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368098a-b4fc-4d83-963b-d4835e7dc8d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.504341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368098a-b4fc-4d83-963b-d4835e7dc8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.504341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368098a-b4fc-4d83-963b-d4835e7dc8d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.505742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368098a-b4fc-4d83-963b-d4835e7dc8d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:06.508101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368098a-b4fc-4d83-963b-d4835e7dc8d1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:06.538722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1196/2384] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:06 2022] POST /identity/v3/users => generated 290 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:06.549466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b1d0b-230c-424c-95f9-34f4fd6da6c4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:06.560002 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1189/2385] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:06 2022] POST /identity/v3/projects => generated 387 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:06.563449 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b1d0b-230c-424c-95f9-34f4fd6da6c4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:06.565086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368098a-b4fc-4d83-963b-d4835e7dc8d1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:06.680483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:06.724222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368098a-b4fc-4d83-963b-d4835e7dc8d1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:06.740291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a221537-0449-40f1-bcf7-0e28bdc6cdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:06.743178 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bcc94-37ba-4d18-b43c-a90f7f210299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:06.743860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a221537-0449-40f1-bcf7-0e28bdc6cdcf admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.744179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a221537-0449-40f1-bcf7-0e28bdc6cdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.744472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a221537-0449-40f1-bcf7-0e28bdc6cdcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.745200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a221537-0449-40f1-bcf7-0e28bdc6cdcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:06.746658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bcc94-37ba-4d18-b43c-a90f7f210299 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.747080 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bcc94-37ba-4d18-b43c-a90f7f210299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.747191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a221537-0449-40f1-bcf7-0e28bdc6cdcf admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:06.749656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7bcc94-37ba-4d18-b43c-a90f7f210299 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.750612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bcc94-37ba-4d18-b43c-a90f7f210299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:06.755648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7bcc94-37ba-4d18-b43c-a90f7f210299 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:06.762587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1197/2386] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:06.766716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a221537-0449-40f1-bcf7-0e28bdc6cdcf admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:06.782954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a221537-0449-40f1-bcf7-0e28bdc6cdcf admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:06.791795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718b216-f56e-499e-adb2-6c5a21c5cc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:06.792592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718b216-f56e-499e-adb2-6c5a21c5cc9e admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.792929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718b216-f56e-499e-adb2-6c5a21c5cc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.793324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718b216-f56e-499e-adb2-6c5a21c5cc9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.796072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718b216-f56e-499e-adb2-6c5a21c5cc9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:06.798302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718b216-f56e-499e-adb2-6c5a21c5cc9e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:06.821374 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1198/2387] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:06.824941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1190/2388] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:06 2022] POST /identity/v3/users => generated 294 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:06.829532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bcc94-37ba-4d18-b43c-a90f7f210299 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:06.832002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718b216-f56e-499e-adb2-6c5a21c5cc9e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:06.852631 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7bcc94-37ba-4d18-b43c-a90f7f210299 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:06.863523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3edc520-8216-4f20-a4c9-3a57f39d94d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:06.865028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edc520-8216-4f20-a4c9-3a57f39d94d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.865424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edc520-8216-4f20-a4c9-3a57f39d94d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.865919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3edc520-8216-4f20-a4c9-3a57f39d94d4 admin admin] PATH_INFO: `/v3/projects/819ff7737bce4ff5978066ccad723d27/users/e3b7626c248f44c68b5129bab1f1232c/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.866894 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718b216-f56e-499e-adb2-6c5a21c5cc9e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:06.881256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a9428-2ee3-4bb6-a77f-4b1c4cf1a713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:06.882100 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a9428-2ee3-4bb6-a77f-4b1c4cf1a713 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.882521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a9428-2ee3-4bb6-a77f-4b1c4cf1a713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.882865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a9428-2ee3-4bb6-a77f-4b1c4cf1a713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.884310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a9428-2ee3-4bb6-a77f-4b1c4cf1a713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:06.887366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a9428-2ee3-4bb6-a77f-4b1c4cf1a713 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:06.915525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1199/2389] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:06.916638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a9428-2ee3-4bb6-a77f-4b1c4cf1a713 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:06.929726 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a9428-2ee3-4bb6-a77f-4b1c4cf1a713 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:06.936654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d6efa-e4c7-4526-814d-12a8f47ccec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:06.937801 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d6efa-e4c7-4526-814d-12a8f47ccec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:06.938234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d6efa-e4c7-4526-814d-12a8f47ccec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:06.940195 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d6efa-e4c7-4526-814d-12a8f47ccec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:06.941222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d6efa-e4c7-4526-814d-12a8f47ccec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:06.943206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d6efa-e4c7-4526-814d-12a8f47ccec0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:06.953238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edc520-8216-4f20-a4c9-3a57f39d94d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=819ff7737bce4ff5978066ccad723d27, user_id=e3b7626c248f44c68b5129bab1f1232c, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:06.956714 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3edc520-8216-4f20-a4c9-3a57f39d94d4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:06.973634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1200/2390] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:06.995141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d6efa-e4c7-4526-814d-12a8f47ccec0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:07.017624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1191/2391] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:06 2022] PUT /identity/v3/projects/819ff7737bce4ff5978066ccad723d27/users/e3b7626c248f44c68b5129bab1f1232c/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:07.017873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d6efa-e4c7-4526-814d-12a8f47ccec0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:07.019797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2308b0-e662-44c0-8b0b-79f188055ba1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.020431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2308b0-e662-44c0-8b0b-79f188055ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.020431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2308b0-e662-44c0-8b0b-79f188055ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.077532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4c2308b0-e662-44c0-8b0b-79f188055ba1 None None] MFA Rules not processed for user `e3b7626c248f44c68b5129bab1f1232c`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:07.079527 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4c2308b0-e662-44c0-8b0b-79f188055ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:07.117835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac73b5-a7ec-4c75-aaef-218b0c7fb5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:07.118540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac73b5-a7ec-4c75-aaef-218b0c7fb5c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.118821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac73b5-a7ec-4c75-aaef-218b0c7fb5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.119101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac73b5-a7ec-4c75-aaef-218b0c7fb5c1 admin admin] PATH_INFO: `/v3/projects/9efb2dff14134f1e9c7d3f9575696b38/users/88aecaf73352499aa1e3bda5b06130d6/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.129308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:07.148621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac73b5-a7ec-4c75-aaef-218b0c7fb5c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9efb2dff14134f1e9c7d3f9575696b38, user_id=88aecaf73352499aa1e3bda5b06130d6, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:07.154238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac73b5-a7ec-4c75-aaef-218b0c7fb5c1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:07.166114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1192/2392] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:30:07.172400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2308b0-e662-44c0-8b0b-79f188055ba1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:07.193833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1201/2393] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:06 2022] PUT /identity/v3/projects/9efb2dff14134f1e9c7d3f9575696b38/users/88aecaf73352499aa1e3bda5b06130d6/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:07.198530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2308b0-e662-44c0-8b0b-79f188055ba1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:07.210821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee5696-dc24-4aec-b90d-321a6feb0ddb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.211613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee5696-dc24-4aec-b90d-321a6feb0ddb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.212087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee5696-dc24-4aec-b90d-321a6feb0ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.272504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d6ee5696-dc24-4aec-b90d-321a6feb0ddb None None] MFA Rules not processed for user `88aecaf73352499aa1e3bda5b06130d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:07.273657 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d6ee5696-dc24-4aec-b90d-321a6feb0ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:07.277045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220490e-edc9-433e-bb42-8d34bf99e9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:07.277963 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220490e-edc9-433e-bb42-8d34bf99e9af admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.278472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220490e-edc9-433e-bb42-8d34bf99e9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.278900 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220490e-edc9-433e-bb42-8d34bf99e9af admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.280524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220490e-edc9-433e-bb42-8d34bf99e9af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:07.282598 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220490e-edc9-433e-bb42-8d34bf99e9af admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:07.325767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1193/2394] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:07 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:07.327568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220490e-edc9-433e-bb42-8d34bf99e9af admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:07.346293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:07.364413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1202/2395] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:07 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:30:07.370138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee5696-dc24-4aec-b90d-321a6feb0ddb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:07.390006 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee5696-dc24-4aec-b90d-321a6feb0ddb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:07.461214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220490e-edc9-433e-bb42-8d34bf99e9af admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:07.466203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f63f29-a889-4e54-9042-68c11eca0704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:07.466822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f63f29-a889-4e54-9042-68c11eca0704 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.467101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f63f29-a889-4e54-9042-68c11eca0704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.467839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f63f29-a889-4e54-9042-68c11eca0704 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.468484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f63f29-a889-4e54-9042-68c11eca0704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:07.470434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f63f29-a889-4e54-9042-68c11eca0704 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:07.488938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07fed2-93d9-414a-8b17-3c78eecc10b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:07.489599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07fed2-93d9-414a-8b17-3c78eecc10b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.489907 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07fed2-93d9-414a-8b17-3c78eecc10b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.490409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f07fed2-93d9-414a-8b17-3c78eecc10b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.491575 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07fed2-93d9-414a-8b17-3c78eecc10b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:07.493788 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f07fed2-93d9-414a-8b17-3c78eecc10b4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:07.494240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1194/2396] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:07 2022] POST /identity/v3/users => generated 296 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:07.498293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f63f29-a889-4e54-9042-68c11eca0704 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:07.516064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f63f29-a889-4e54-9042-68c11eca0704 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:07.521994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f6e7d-2404-4c50-a763-52e0cfce9bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:07.522808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f6e7d-2404-4c50-a763-52e0cfce9bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.523236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f6e7d-2404-4c50-a763-52e0cfce9bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.523754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f6e7d-2404-4c50-a763-52e0cfce9bb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.524493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f6e7d-2404-4c50-a763-52e0cfce9bb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:07.526367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f6e7d-2404-4c50-a763-52e0cfce9bb3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:07.542621 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1195/2397] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:07.544893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f6e7d-2404-4c50-a763-52e0cfce9bb3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:07.549800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1203/2398] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:07 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:07.554084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07fed2-93d9-414a-8b17-3c78eecc10b4 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:07.561516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f6e7d-2404-4c50-a763-52e0cfce9bb3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:07.646989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f07fed2-93d9-414a-8b17-3c78eecc10b4 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:07.652621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea943630-5950-43c6-bb9f-07cf0b7fc628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:07.653312 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea943630-5950-43c6-bb9f-07cf0b7fc628 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.653596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea943630-5950-43c6-bb9f-07cf0b7fc628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.654094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea943630-5950-43c6-bb9f-07cf0b7fc628 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.655474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea943630-5950-43c6-bb9f-07cf0b7fc628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:07.657934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea943630-5950-43c6-bb9f-07cf0b7fc628 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:07.662640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0efab5-8f43-4ea8-950d-2498093caeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:07.663534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0efab5-8f43-4ea8-950d-2498093caeb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.663988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0efab5-8f43-4ea8-950d-2498093caeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.664461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0efab5-8f43-4ea8-950d-2498093caeb1 admin admin] PATH_INFO: `/v3/projects/fd5e1942a9ee4026a37760f2f2ad1ffe/users/c639b58e382e4694a1a6f8a17e13a92d/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.668583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:07.682535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0efab5-8f43-4ea8-950d-2498093caeb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd5e1942a9ee4026a37760f2f2ad1ffe, user_id=c639b58e382e4694a1a6f8a17e13a92d, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:07.685014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1204/2399] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:07 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:07.685214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0efab5-8f43-4ea8-950d-2498093caeb1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:07.688903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea943630-5950-43c6-bb9f-07cf0b7fc628 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:07.699412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1196/2400] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:07 2022] PUT /identity/v3/projects/fd5e1942a9ee4026a37760f2f2ad1ffe/users/c639b58e382e4694a1a6f8a17e13a92d/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:07.702898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea943630-5950-43c6-bb9f-07cf0b7fc628 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:07.703971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0efab5-8f43-4ea8-950d-2498093caeb1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:07.783876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8844775-c859-4a41-a909-292cfa19bb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:07.784727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8844775-c859-4a41-a909-292cfa19bb78 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.785410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8844775-c859-4a41-a909-292cfa19bb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.785688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8844775-c859-4a41-a909-292cfa19bb78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.786525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8844775-c859-4a41-a909-292cfa19bb78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:07.788123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8844775-c859-4a41-a909-292cfa19bb78 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:07.802396 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1205/2401] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:07.802629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0efab5-8f43-4ea8-950d-2498093caeb1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:07.804196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8844775-c859-4a41-a909-292cfa19bb78 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:07.826307 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8844775-c859-4a41-a909-292cfa19bb78 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:07.837909 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305bbc6-a321-43cf-b9ad-8bc70b2ab6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:07.838314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ff9fa-741d-4310-800a-3a20910aa060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:07.838587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305bbc6-a321-43cf-b9ad-8bc70b2ab6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.838864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305bbc6-a321-43cf-b9ad-8bc70b2ab6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.839142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305bbc6-a321-43cf-b9ad-8bc70b2ab6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.839298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ff9fa-741d-4310-800a-3a20910aa060 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.839674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ff9fa-741d-4310-800a-3a20910aa060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.840031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305bbc6-a321-43cf-b9ad-8bc70b2ab6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:07.840145 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ff9fa-741d-4310-800a-3a20910aa060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.840851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ff9fa-741d-4310-800a-3a20910aa060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:07.841664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305bbc6-a321-43cf-b9ad-8bc70b2ab6d5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:07.844750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ff9fa-741d-4310-800a-3a20910aa060 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:07.870720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1206/2402] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:07.870948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1197/2403] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:07 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:07.872827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ff9fa-741d-4310-800a-3a20910aa060 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:07.874460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305bbc6-a321-43cf-b9ad-8bc70b2ab6d5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:07.885005 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ff9fa-741d-4310-800a-3a20910aa060 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:07.887770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305bbc6-a321-43cf-b9ad-8bc70b2ab6d5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:07.890736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81faccc-71bd-4400-bc29-1aaa9d764edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:07.891418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81faccc-71bd-4400-bc29-1aaa9d764edc admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.894396 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81faccc-71bd-4400-bc29-1aaa9d764edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.895098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81faccc-71bd-4400-bc29-1aaa9d764edc admin admin] PATH_INFO: `/v3/projects/52d6d2c1d9424eca8e36194320c24bde/users/83099aa35c3d482591c580f0510854f3/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.908756 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a9eb3-166f-4644-bba7-696652b76e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:07.909445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a9eb3-166f-4644-bba7-696652b76e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.909722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a9eb3-166f-4644-bba7-696652b76e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.909996 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a9eb3-166f-4644-bba7-696652b76e68 admin admin] PATH_INFO: `/v3/projects/fd5e1942a9ee4026a37760f2f2ad1ffe/users/c639b58e382e4694a1a6f8a17e13a92d/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.916150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81faccc-71bd-4400-bc29-1aaa9d764edc admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d6d2c1d9424eca8e36194320c24bde, user_id=83099aa35c3d482591c580f0510854f3, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:07.921414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81faccc-71bd-4400-bc29-1aaa9d764edc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:07.929268 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a9eb3-166f-4644-bba7-696652b76e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd5e1942a9ee4026a37760f2f2ad1ffe, user_id=c639b58e382e4694a1a6f8a17e13a92d, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:07.942900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1207/2404] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:07 2022] PUT /identity/v3/projects/52d6d2c1d9424eca8e36194320c24bde/users/83099aa35c3d482591c580f0510854f3/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:07.951532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a9eb3-166f-4644-bba7-696652b76e68 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:07.957003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344473b5-67f6-494d-9c49-eed897b60200 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.958335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344473b5-67f6-494d-9c49-eed897b60200 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.958582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:07.958805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344473b5-67f6-494d-9c49-eed897b60200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.974809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1198/2405] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:07 2022] PUT /identity/v3/projects/fd5e1942a9ee4026a37760f2f2ad1ffe/users/c639b58e382e4694a1a6f8a17e13a92d/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:07.988059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff297fef-5ed4-4c07-9056-674f29fe7903 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:07.988345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff297fef-5ed4-4c07-9056-674f29fe7903 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:07.988623 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff297fef-5ed4-4c07-9056-674f29fe7903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:07.999252 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-344473b5-67f6-494d-9c49-eed897b60200 None None] MFA Rules not processed for user `83099aa35c3d482591c580f0510854f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:08.000755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-344473b5-67f6-494d-9c49-eed897b60200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:08.053312 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ff297fef-5ed4-4c07-9056-674f29fe7903 None None] MFA Rules not processed for user `c639b58e382e4694a1a6f8a17e13a92d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:08.055234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ff297fef-5ed4-4c07-9056-674f29fe7903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:08.116132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1208/2406] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:30:08.120839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344473b5-67f6-494d-9c49-eed897b60200 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:08.214179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1199/2407] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:08.233875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff297fef-5ed4-4c07-9056-674f29fe7903 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:08.235326 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ff297fef-5ed4-4c07-9056-674f29fe7903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:08.256034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:08.303514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344473b5-67f6-494d-9c49-eed897b60200 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:08.308809 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ba4ec-f6d9-4577-a2d1-c062bda98af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:08.309512 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ba4ec-f6d9-4577-a2d1-c062bda98af3 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:08.309873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ba4ec-f6d9-4577-a2d1-c062bda98af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:08.310238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ba4ec-f6d9-4577-a2d1-c062bda98af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:08.310899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ba4ec-f6d9-4577-a2d1-c062bda98af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:08.312831 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ba4ec-f6d9-4577-a2d1-c062bda98af3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:08.328532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff297fef-5ed4-4c07-9056-674f29fe7903 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:08.333853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21e0ee-d153-454b-8385-fd76d69d5488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:08.334661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21e0ee-d153-454b-8385-fd76d69d5488 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:08.335079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21e0ee-d153-454b-8385-fd76d69d5488 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:08.335440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21e0ee-d153-454b-8385-fd76d69d5488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:08.347683 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21e0ee-d153-454b-8385-fd76d69d5488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:08.348171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1209/2408] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:08 2022] POST /identity/v3/projects => generated 387 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:08.349766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21e0ee-d153-454b-8385-fd76d69d5488 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:08.361269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ba4ec-f6d9-4577-a2d1-c062bda98af3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:08.362098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1200/2409] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:30:08.484174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ba4ec-f6d9-4577-a2d1-c062bda98af3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:08.496204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b60e2c-8673-4ba5-ae36-cab138d171e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:08.496204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b60e2c-8673-4ba5-ae36-cab138d171e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:08.496204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b60e2c-8673-4ba5-ae36-cab138d171e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:08.496204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b60e2c-8673-4ba5-ae36-cab138d171e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:08.496790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b60e2c-8673-4ba5-ae36-cab138d171e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:08.496790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b60e2c-8673-4ba5-ae36-cab138d171e8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:08.540279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1210/2410] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:08 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:08.543301 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21e0ee-d153-454b-8385-fd76d69d5488 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:08.568726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21e0ee-d153-454b-8385-fd76d69d5488 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:08.577839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9da25-0247-4598-8114-3df26c05e83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:08.578648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9da25-0247-4598-8114-3df26c05e83e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:08.578928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9da25-0247-4598-8114-3df26c05e83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:08.579200 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9da25-0247-4598-8114-3df26c05e83e admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:08.581649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9da25-0247-4598-8114-3df26c05e83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:08.581649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a9da25-0247-4598-8114-3df26c05e83e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:08.593347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1201/2411] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:08.598684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b60e2c-8673-4ba5-ae36-cab138d171e8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:08.610584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b60e2c-8673-4ba5-ae36-cab138d171e8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:08.616412 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c8f7e-392e-404b-9960-4dfbe555b4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:08.617092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c8f7e-392e-404b-9960-4dfbe555b4b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:08.617375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c8f7e-392e-404b-9960-4dfbe555b4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:08.617682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c8f7e-392e-404b-9960-4dfbe555b4b5 admin admin] PATH_INFO: `/v3/projects/ef4d9846ff5a48d19f7c3010f8bfe832/users/d1c4658a598c44df9d81542c80914da5/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:08.650855 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c8f7e-392e-404b-9960-4dfbe555b4b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4d9846ff5a48d19f7c3010f8bfe832, user_id=d1c4658a598c44df9d81542c80914da5, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:08.653631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c8f7e-392e-404b-9960-4dfbe555b4b5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:08.672076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1211/2412] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:08 2022] PUT /identity/v3/projects/ef4d9846ff5a48d19f7c3010f8bfe832/users/d1c4658a598c44df9d81542c80914da5/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:08.676227 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9da25-0247-4598-8114-3df26c05e83e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:08.687323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9da25-0247-4598-8114-3df26c05e83e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:08.789373 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dcf888-f12d-4ab9-bcef-d0f1430dc7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:08.790005 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcf888-f12d-4ab9-bcef-d0f1430dc7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:08.790389 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcf888-f12d-4ab9-bcef-d0f1430dc7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:08.790716 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dcf888-f12d-4ab9-bcef-d0f1430dc7e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:08.791432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dcf888-f12d-4ab9-bcef-d0f1430dc7e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:08.792168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c8f7e-392e-404b-9960-4dfbe555b4b5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:08.793201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dcf888-f12d-4ab9-bcef-d0f1430dc7e5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:08.804413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1202/2413] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:08 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:08.809703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dcf888-f12d-4ab9-bcef-d0f1430dc7e5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:08.821102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dcf888-f12d-4ab9-bcef-d0f1430dc7e5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:08.835935 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08caff62-006d-4e18-89fc-e81f18f786b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:08.836635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caff62-006d-4e18-89fc-e81f18f786b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:08.836915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caff62-006d-4e18-89fc-e81f18f786b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:08.837186 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08caff62-006d-4e18-89fc-e81f18f786b4 admin admin] PATH_INFO: `/v3/projects/ef4d9846ff5a48d19f7c3010f8bfe832/users/d1c4658a598c44df9d81542c80914da5/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:08.842515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caff62-006d-4e18-89fc-e81f18f786b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4d9846ff5a48d19f7c3010f8bfe832, user_id=d1c4658a598c44df9d81542c80914da5, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:08.847681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08caff62-006d-4e18-89fc-e81f18f786b4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:08.851630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:08.870152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1203/2414] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:08 2022] PUT /identity/v3/projects/ef4d9846ff5a48d19f7c3010f8bfe832/users/d1c4658a598c44df9d81542c80914da5/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:08.874971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6249d-27e3-4eab-afd4-6181ed2eb68a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:08.875256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6249d-27e3-4eab-afd4-6181ed2eb68a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:08.875531 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e6249d-27e3-4eab-afd4-6181ed2eb68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:08.899096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c8f7e-392e-404b-9960-4dfbe555b4b5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:08.910819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-09e6249d-27e3-4eab-afd4-6181ed2eb68a None None] MFA Rules not processed for user `d1c4658a598c44df9d81542c80914da5`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:08.912069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-09e6249d-27e3-4eab-afd4-6181ed2eb68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:08.964583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:08.989197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d7c77-d065-47ca-86ff-5b77d87de21c tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c639b58e382e4694a1a6f8a17e13a92d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd5e1942a9ee4026a37760f2f2ad1ffe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:08.990096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d7c77-d065-47ca-86ff-5b77d87de21c tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:08.990261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d7c77-d065-47ca-86ff-5b77d87de21c tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:08.990537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108d7c77-d065-47ca-86ff-5b77d87de21c tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:08.991194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d7c77-d065-47ca-86ff-5b77d87de21c tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:08.991505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d7c77-d065-47ca-86ff-5b77d87de21c tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:08.993112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108d7c77-d065-47ca-86ff-5b77d87de21c tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:09.005427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1204/2415] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:30:09.009311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1212/2416] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:30:09.010432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6249d-27e3-4eab-afd4-6181ed2eb68a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:09.023095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e6249d-27e3-4eab-afd4-6181ed2eb68a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:09.028023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345c949-7ca5-402d-b644-48e8e1e2f5e2 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c639b58e382e4694a1a6f8a17e13a92d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd5e1942a9ee4026a37760f2f2ad1ffe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:09.028649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345c949-7ca5-402d-b644-48e8e1e2f5e2 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:09.028941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345c949-7ca5-402d-b644-48e8e1e2f5e2 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:09.029219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345c949-7ca5-402d-b644-48e8e1e2f5e2 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:09.029927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345c949-7ca5-402d-b644-48e8e1e2f5e2 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:09.037836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d7c77-d065-47ca-86ff-5b77d87de21c tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:09.040095 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-108d7c77-d065-47ca-86ff-5b77d87de21c tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:09.042058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345c949-7ca5-402d-b644-48e8e1e2f5e2 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:09.098037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1205/2417] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:09 2022] POST /identity/v3/projects => generated 353 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:09.100740 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345c949-7ca5-402d-b644-48e8e1e2f5e2 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:09.133489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108d7c77-d065-47ca-86ff-5b77d87de21c tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:09.230580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345c949-7ca5-402d-b644-48e8e1e2f5e2 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:09.230810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25051b8f-b2e7-4af8-badd-226f1b0ecab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:09.231409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25051b8f-b2e7-4af8-badd-226f1b0ecab4 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:09.231689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25051b8f-b2e7-4af8-badd-226f1b0ecab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:09.231964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25051b8f-b2e7-4af8-badd-226f1b0ecab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:09.237144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c770c1-08c6-4f11-aeb9-0ab0b6393657 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c639b58e382e4694a1a6f8a17e13a92d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd5e1942a9ee4026a37760f2f2ad1ffe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:09.237785 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c770c1-08c6-4f11-aeb9-0ab0b6393657 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:09.238075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c770c1-08c6-4f11-aeb9-0ab0b6393657 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:09.238362 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c770c1-08c6-4f11-aeb9-0ab0b6393657 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:09.239026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c770c1-08c6-4f11-aeb9-0ab0b6393657 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:09.239336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c770c1-08c6-4f11-aeb9-0ab0b6393657 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:09.240939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c770c1-08c6-4f11-aeb9-0ab0b6393657 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:09.253763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1206/2418] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:30:09.296833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c770c1-08c6-4f11-aeb9-0ab0b6393657 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:09.309274 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c770c1-08c6-4f11-aeb9-0ab0b6393657 tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:09.316082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd8ec9-c372-43c3-b23a-18912dcf1eff tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c639b58e382e4694a1a6f8a17e13a92d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fd5e1942a9ee4026a37760f2f2ad1ffe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:09.317242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd8ec9-c372-43c3-b23a-18912dcf1eff tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:09.317934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd8ec9-c372-43c3-b23a-18912dcf1eff tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:09.318672 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd8ec9-c372-43c3-b23a-18912dcf1eff tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] PATH_INFO: `/v3/projects/d8a39270328a49a9b72fed5cc0ed1639` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:09.319388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25051b8f-b2e7-4af8-badd-226f1b0ecab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:09.320823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd8ec9-c372-43c3-b23a-18912dcf1eff tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d8a39270328a49a9b72fed5cc0ed1639)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:09.320967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25051b8f-b2e7-4af8-badd-226f1b0ecab4 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:09.323226 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd8ec9-c372-43c3-b23a-18912dcf1eff tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:09.342955 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1213/2419] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:30:09.425721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:09.431677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-05cd8ec9-c372-43c3-b23a-18912dcf1eff tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a39270328a49a9b72fed5cc0ed1639', 'request_id': 'req-05cd8ec9-c372-43c3-b23a-18912dcf1eff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:09.457769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-05cd8ec9-c372-43c3-b23a-18912dcf1eff tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a39270328a49a9b72fed5cc0ed1639', 'request_id': 'req-05cd8ec9-c372-43c3-b23a-18912dcf1eff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:09.465950 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-05cd8ec9-c372-43c3-b23a-18912dcf1eff tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a39270328a49a9b72fed5cc0ed1639', 'request_id': 'req-05cd8ec9-c372-43c3-b23a-18912dcf1eff'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:09.499265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-05cd8ec9-c372-43c3-b23a-18912dcf1eff tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] The token cache is being invalidate because project d8a39270328a49a9b72fed5cc0ed1639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:09.499557 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-05cd8ec9-c372-43c3-b23a-18912dcf1eff tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:09.502439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-05cd8ec9-c372-43c3-b23a-18912dcf1eff tempest-VolumeQuotaClassesTest-392132876 tempest-VolumeQuotaClassesTest-392132876-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:09.504381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1207/2420] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:09 2022] DELETE /identity/v3/projects/d8a39270328a49a9b72fed5cc0ed1639 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:09.779410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25051b8f-b2e7-4af8-badd-226f1b0ecab4 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:09.839964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:09.871518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25051b8f-b2e7-4af8-badd-226f1b0ecab4 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:10.019429 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:10.021227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:10.021799 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:10.022941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] PATH_INFO: `/v3/users/e3b7626c248f44c68b5129bab1f1232c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:10.024750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3b7626c248f44c68b5129bab1f1232c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:10.028166 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:10.143187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3b7626c248f44c68b5129bab1f1232c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:10.175810 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3b7626c248f44c68b5129bab1f1232c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:10.176308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3b7626c248f44c68b5129bab1f1232c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:10.213434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3b7626c248f44c68b5129bab1f1232c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:10.245847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3b7626c248f44c68b5129bab1f1232c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:10.258705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1214/2421] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:09 2022] DELETE /identity/v3/users/e3b7626c248f44c68b5129bab1f1232c => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:10.264272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9e3b1-7d44-4d7a-a43d-40ce0a97e5c3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:10.265073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9e3b1-7d44-4d7a-a43d-40ce0a97e5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:10.265073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9e3b1-7d44-4d7a-a43d-40ce0a97e5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:10.322539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bca9e3b1-7d44-4d7a-a43d-40ce0a97e5c3 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:10.324229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bca9e3b1-7d44-4d7a-a43d-40ce0a97e5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:10.427715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1208/2422] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:10.441751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:10.443354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:10.596690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e52bd-7eff-4406-9939-0d1b3b210f98 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:10.616019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f81237-1169-4bb6-ace7-fc7575005a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:10.616676 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f81237-1169-4bb6-ace7-fc7575005a96 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:10.616962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f81237-1169-4bb6-ace7-fc7575005a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:10.617239 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f81237-1169-4bb6-ace7-fc7575005a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:10.632450 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f81237-1169-4bb6-ace7-fc7575005a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:10.634290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f81237-1169-4bb6-ace7-fc7575005a96 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:10.648213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1215/2423] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:10.669865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9e3b1-7d44-4d7a-a43d-40ce0a97e5c3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:10.672063 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bca9e3b1-7d44-4d7a-a43d-40ce0a97e5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:10.782518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9e3b1-7d44-4d7a-a43d-40ce0a97e5c3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:10.786125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879a91b-3f99-4496-a5a9-f02feaadc3ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:10.786736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879a91b-3f99-4496-a5a9-f02feaadc3ba service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:10.787017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879a91b-3f99-4496-a5a9-f02feaadc3ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:10.787452 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879a91b-3f99-4496-a5a9-f02feaadc3ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:10.789334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5879a91b-3f99-4496-a5a9-f02feaadc3ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:10.878286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879a91b-3f99-4496-a5a9-f02feaadc3ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:10.879743 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879a91b-3f99-4496-a5a9-f02feaadc3ba service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:10.890658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1209/2424] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:11.123878 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f81237-1169-4bb6-ace7-fc7575005a96 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:11.125405 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-07f81237-1169-4bb6-ace7-fc7575005a96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:11.159074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f81237-1169-4bb6-ace7-fc7575005a96 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:11.185719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:11.188001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:11.188714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:11.189204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] PATH_INFO: `/v3/projects/819ff7737bce4ff5978066ccad723d27` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:11.191099 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=819ff7737bce4ff5978066ccad723d27)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:11.200619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:11.204237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:11.333647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819ff7737bce4ff5978066ccad723d27', 'request_id': 'req-0883f616-22b7-4827-9f83-f0a6f32e7021'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:11.344708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819ff7737bce4ff5978066ccad723d27', 'request_id': 'req-0883f616-22b7-4827-9f83-f0a6f32e7021'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:11.359111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819ff7737bce4ff5978066ccad723d27', 'request_id': 'req-0883f616-22b7-4827-9f83-f0a6f32e7021'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:11.372849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] The token cache is being invalidate because project 819ff7737bce4ff5978066ccad723d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:11.373325 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:11.374069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:11.382755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1216/2425] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:11 2022] DELETE /identity/v3/projects/819ff7737bce4ff5978066ccad723d27 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:11.389657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879a91b-3f99-4496-a5a9-f02feaadc3ba service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:11.402566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879a91b-3f99-4496-a5a9-f02feaadc3ba service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:11.443541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:11.482829 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76ab7-e223-43dd-a793-8f7587c16209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:11.483797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76ab7-e223-43dd-a793-8f7587c16209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:11.484088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76ab7-e223-43dd-a793-8f7587c16209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:11.484352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76ab7-e223-43dd-a793-8f7587c16209 admin admin] PATH_INFO: `/v3/users/c639b58e382e4694a1a6f8a17e13a92d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:11.485523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca76ab7-e223-43dd-a793-8f7587c16209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c639b58e382e4694a1a6f8a17e13a92d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:11.487499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca76ab7-e223-43dd-a793-8f7587c16209 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:11.557027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cca76ab7-e223-43dd-a793-8f7587c16209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c639b58e382e4694a1a6f8a17e13a92d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:11.568782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cca76ab7-e223-43dd-a793-8f7587c16209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c639b58e382e4694a1a6f8a17e13a92d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:11.595791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cca76ab7-e223-43dd-a793-8f7587c16209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c639b58e382e4694a1a6f8a17e13a92d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:11.596733 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cca76ab7-e223-43dd-a793-8f7587c16209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c639b58e382e4694a1a6f8a17e13a92d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:11.606471 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cca76ab7-e223-43dd-a793-8f7587c16209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c639b58e382e4694a1a6f8a17e13a92d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:11.612688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1210/2426] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:11 2022] DELETE /identity/v3/users/c639b58e382e4694a1a6f8a17e13a92d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:12.194782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:12.197796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:12.224132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0883f616-22b7-4827-9f83-f0a6f32e7021 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:12.346119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c414e3bf-3128-4157-8440-24c24468b863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:12.346119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414e3bf-3128-4157-8440-24c24468b863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:12.346119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414e3bf-3128-4157-8440-24c24468b863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:12.346697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c414e3bf-3128-4157-8440-24c24468b863 admin admin] PATH_INFO: `/v3/projects/fd5e1942a9ee4026a37760f2f2ad1ffe` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:12.347856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414e3bf-3128-4157-8440-24c24468b863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd5e1942a9ee4026a37760f2f2ad1ffe)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:12.350027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c414e3bf-3128-4157-8440-24c24468b863 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:12.449838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c414e3bf-3128-4157-8440-24c24468b863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd5e1942a9ee4026a37760f2f2ad1ffe', 'request_id': 'req-c414e3bf-3128-4157-8440-24c24468b863'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:12.471759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c414e3bf-3128-4157-8440-24c24468b863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd5e1942a9ee4026a37760f2f2ad1ffe', 'request_id': 'req-c414e3bf-3128-4157-8440-24c24468b863'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:12.493273 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c414e3bf-3128-4157-8440-24c24468b863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd5e1942a9ee4026a37760f2f2ad1ffe', 'request_id': 'req-c414e3bf-3128-4157-8440-24c24468b863'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:12.506603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c414e3bf-3128-4157-8440-24c24468b863 admin admin] The token cache is being invalidate because project fd5e1942a9ee4026a37760f2f2ad1ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:12.507824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c414e3bf-3128-4157-8440-24c24468b863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:12.508637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c414e3bf-3128-4157-8440-24c24468b863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:12.513854 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1217/2427] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:12 2022] DELETE /identity/v3/projects/fd5e1942a9ee4026a37760f2f2ad1ffe => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:12.515661 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be95bc-69ce-4f88-b48a-a5906fce2a51 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:12.516164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be95bc-69ce-4f88-b48a-a5906fce2a51 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:12.516261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be95bc-69ce-4f88-b48a-a5906fce2a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:12.549495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-36be95bc-69ce-4f88-b48a-a5906fce2a51 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:12.550014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-36be95bc-69ce-4f88-b48a-a5906fce2a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:12.714101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1211/2428] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:12.721255 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559267fd-f303-4bd3-908c-b99d7fd56dbd None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:12.721627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559267fd-f303-4bd3-908c-b99d7fd56dbd None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:12.721974 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559267fd-f303-4bd3-908c-b99d7fd56dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:12.743333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-559267fd-f303-4bd3-908c-b99d7fd56dbd None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:12.745007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-559267fd-f303-4bd3-908c-b99d7fd56dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:12.758340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1218/2429] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:12.763056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be95bc-69ce-4f88-b48a-a5906fce2a51 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:12.774496 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be95bc-69ce-4f88-b48a-a5906fce2a51 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:12.781590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749575fe-f898-4deb-9886-e0cd1b29bcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:12.782394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749575fe-f898-4deb-9886-e0cd1b29bcf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:12.782678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749575fe-f898-4deb-9886-e0cd1b29bcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:12.782947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749575fe-f898-4deb-9886-e0cd1b29bcf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:12.783614 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749575fe-f898-4deb-9886-e0cd1b29bcf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:12.783906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749575fe-f898-4deb-9886-e0cd1b29bcf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:12.785503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749575fe-f898-4deb-9886-e0cd1b29bcf1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:12.803300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1212/2430] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:30:12.804473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559267fd-f303-4bd3-908c-b99d7fd56dbd None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:12.820387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559267fd-f303-4bd3-908c-b99d7fd56dbd None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:12.826911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f7692-22cf-4e5a-a9b1-5b7e2b044863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:12.827538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f7692-22cf-4e5a-a9b1-5b7e2b044863 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:12.828933 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f7692-22cf-4e5a-a9b1-5b7e2b044863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:12.828933 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f7692-22cf-4e5a-a9b1-5b7e2b044863 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:12.829314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f7692-22cf-4e5a-a9b1-5b7e2b044863 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:12.831452 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f7692-22cf-4e5a-a9b1-5b7e2b044863 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:12.850181 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:12.873913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1219/2431] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:12 2022] POST /identity/v3/projects => generated 363 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:12.884420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749575fe-f898-4deb-9886-e0cd1b29bcf1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:12.903347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749575fe-f898-4deb-9886-e0cd1b29bcf1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:12.999990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:13.035237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68581541-8d8c-4909-a89d-1fb8d820e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:13.036570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68581541-8d8c-4909-a89d-1fb8d820e86f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:13.037204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68581541-8d8c-4909-a89d-1fb8d820e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:13.037788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68581541-8d8c-4909-a89d-1fb8d820e86f admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:13.039139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68581541-8d8c-4909-a89d-1fb8d820e86f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:13.060364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68581541-8d8c-4909-a89d-1fb8d820e86f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:13.110257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1213/2432] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:12 2022] POST /identity/v3/users => generated 282 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:13.111092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f7692-22cf-4e5a-a9b1-5b7e2b044863 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:13.132167 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f7692-22cf-4e5a-a9b1-5b7e2b044863 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:13.138077 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0e734-35f3-4319-baf7-8e5103ee62f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:13.139527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e734-35f3-4319-baf7-8e5103ee62f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:13.139865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e734-35f3-4319-baf7-8e5103ee62f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:13.140210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e734-35f3-4319-baf7-8e5103ee62f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:13.141345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0e734-35f3-4319-baf7-8e5103ee62f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:13.143226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f0e734-35f3-4319-baf7-8e5103ee62f1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:13.166720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1220/2433] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:13.168023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68581541-8d8c-4909-a89d-1fb8d820e86f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:13.183528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68581541-8d8c-4909-a89d-1fb8d820e86f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:13.202452 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f2537-eb43-4bbd-a9fa-862f20959f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:13.203326 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f2537-eb43-4bbd-a9fa-862f20959f6a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:13.203673 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f2537-eb43-4bbd-a9fa-862f20959f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:13.203892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f2537-eb43-4bbd-a9fa-862f20959f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:13.204736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f2537-eb43-4bbd-a9fa-862f20959f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:13.206653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f2537-eb43-4bbd-a9fa-862f20959f6a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:13.222195 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1214/2434] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:13.223332 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0e734-35f3-4319-baf7-8e5103ee62f1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:13.237336 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f0e734-35f3-4319-baf7-8e5103ee62f1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:13.246727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b8dd2-6272-4fb0-b90c-35519b5c4e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:13.247722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b8dd2-6272-4fb0-b90c-35519b5c4e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:13.249134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b8dd2-6272-4fb0-b90c-35519b5c4e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:13.255617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b8dd2-6272-4fb0-b90c-35519b5c4e7e admin admin] PATH_INFO: `/v3/projects/3fd26d388dda4c6797e8324b7632bccd/users/b4d5e044f5b041b7a2404b3ae25bde01/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:13.300601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b8dd2-6272-4fb0-b90c-35519b5c4e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd26d388dda4c6797e8324b7632bccd, user_id=b4d5e044f5b041b7a2404b3ae25bde01, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:13.303718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b8dd2-6272-4fb0-b90c-35519b5c4e7e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:13.339754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1221/2435] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:13 2022] PUT /identity/v3/projects/3fd26d388dda4c6797e8324b7632bccd/users/b4d5e044f5b041b7a2404b3ae25bde01/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:13.350839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55276ff8-674b-4614-99bc-c64e02e916ae None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:13.351492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55276ff8-674b-4614-99bc-c64e02e916ae None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:13.352031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55276ff8-674b-4614-99bc-c64e02e916ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:13.394509 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-55276ff8-674b-4614-99bc-c64e02e916ae None None] MFA Rules not processed for user `b4d5e044f5b041b7a2404b3ae25bde01`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:13.395915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-55276ff8-674b-4614-99bc-c64e02e916ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:13.498808 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1215/2436] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:13.505755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b8dd2-6272-4fb0-b90c-35519b5c4e7e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:13.598247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b8dd2-6272-4fb0-b90c-35519b5c4e7e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:13.605596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620161f-d9fa-49b3-8787-c68d904e887a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:13.606203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620161f-d9fa-49b3-8787-c68d904e887a admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:13.606486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620161f-d9fa-49b3-8787-c68d904e887a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:13.606762 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620161f-d9fa-49b3-8787-c68d904e887a admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:13.608086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620161f-d9fa-49b3-8787-c68d904e887a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:13.610196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620161f-d9fa-49b3-8787-c68d904e887a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:13.656082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1222/2437] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:13 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:13.657569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55276ff8-674b-4614-99bc-c64e02e916ae None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:13.671596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55276ff8-674b-4614-99bc-c64e02e916ae None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:13.736098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26338d-aaf8-4442-bb31-5d4bcf40608b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:13.737654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26338d-aaf8-4442-bb31-5d4bcf40608b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:13.737654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26338d-aaf8-4442-bb31-5d4bcf40608b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:13.737858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26338d-aaf8-4442-bb31-5d4bcf40608b admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:13.738670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26338d-aaf8-4442-bb31-5d4bcf40608b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:13.741592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26338d-aaf8-4442-bb31-5d4bcf40608b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:13.778585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1216/2438] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:13 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:13.781464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620161f-d9fa-49b3-8787-c68d904e887a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:13.799268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620161f-d9fa-49b3-8787-c68d904e887a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:13.803995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da74180-1ec2-44a2-8b4e-d5e861748ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:13.804603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da74180-1ec2-44a2-8b4e-d5e861748ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:13.804881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da74180-1ec2-44a2-8b4e-d5e861748ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:13.805152 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da74180-1ec2-44a2-8b4e-d5e861748ef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:13.805826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da74180-1ec2-44a2-8b4e-d5e861748ef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:13.807489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da74180-1ec2-44a2-8b4e-d5e861748ef7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:13.823630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1223/2439] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:13.825655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26338d-aaf8-4442-bb31-5d4bcf40608b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:13.836594 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26338d-aaf8-4442-bb31-5d4bcf40608b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:13.843352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5d85f-dd31-4921-9cef-21187949fd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:13.844011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5d85f-dd31-4921-9cef-21187949fd7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:13.844296 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5d85f-dd31-4921-9cef-21187949fd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:13.844584 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5d85f-dd31-4921-9cef-21187949fd7f admin admin] PATH_INFO: `/v3/projects/937f1c4dfd964cc08a019679f729fefb/users/de9e3a22c33c49b18bfe308c88181a72/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:13.862590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5d85f-dd31-4921-9cef-21187949fd7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=937f1c4dfd964cc08a019679f729fefb, user_id=de9e3a22c33c49b18bfe308c88181a72, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:13.865394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5d85f-dd31-4921-9cef-21187949fd7f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:13.879529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1217/2440] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:13 2022] PUT /identity/v3/projects/937f1c4dfd964cc08a019679f729fefb/users/de9e3a22c33c49b18bfe308c88181a72/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:13.884092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da74180-1ec2-44a2-8b4e-d5e861748ef7 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:13.895547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da74180-1ec2-44a2-8b4e-d5e861748ef7 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:13.970363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f926f378-7016-4e40-bd76-5963e3c5ee54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:13.971025 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926f378-7016-4e40-bd76-5963e3c5ee54 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:13.971305 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926f378-7016-4e40-bd76-5963e3c5ee54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:13.971688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f926f378-7016-4e40-bd76-5963e3c5ee54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:13.972229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926f378-7016-4e40-bd76-5963e3c5ee54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:13.973808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f926f378-7016-4e40-bd76-5963e3c5ee54 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:13.985871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1224/2441] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:13 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:13.989781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5d85f-dd31-4921-9cef-21187949fd7f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:14.001027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5d85f-dd31-4921-9cef-21187949fd7f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:14.005788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628dc8e-fe51-4dc0-9c88-266bb0b985c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:14.007050 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628dc8e-fe51-4dc0-9c88-266bb0b985c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:14.007050 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628dc8e-fe51-4dc0-9c88-266bb0b985c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:14.007050 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628dc8e-fe51-4dc0-9c88-266bb0b985c6 admin admin] PATH_INFO: `/v3/projects/937f1c4dfd964cc08a019679f729fefb/users/de9e3a22c33c49b18bfe308c88181a72/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:14.009524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628dc8e-fe51-4dc0-9c88-266bb0b985c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=937f1c4dfd964cc08a019679f729fefb, user_id=de9e3a22c33c49b18bfe308c88181a72, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:14.012712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628dc8e-fe51-4dc0-9c88-266bb0b985c6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:14.022205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:14.030187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1218/2442] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:13 2022] PUT /identity/v3/projects/937f1c4dfd964cc08a019679f729fefb/users/de9e3a22c33c49b18bfe308c88181a72/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:14.041596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186226f7-8dde-4d3e-877c-a48690344276 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:14.041982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186226f7-8dde-4d3e-877c-a48690344276 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:14.042474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186226f7-8dde-4d3e-877c-a48690344276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:14.073892 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-186226f7-8dde-4d3e-877c-a48690344276 None None] MFA Rules not processed for user `de9e3a22c33c49b18bfe308c88181a72`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:14.075148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-186226f7-8dde-4d3e-877c-a48690344276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:14.094642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:14.156483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1225/2443] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:14.168788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628dc8e-fe51-4dc0-9c88-266bb0b985c6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:14.170974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d628dc8e-fe51-4dc0-9c88-266bb0b985c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:14.244641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628dc8e-fe51-4dc0-9c88-266bb0b985c6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:14.253605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e93ebf-a55d-456b-bcfd-e5a8458f8489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:14.255059 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e93ebf-a55d-456b-bcfd-e5a8458f8489 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:14.255450 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e93ebf-a55d-456b-bcfd-e5a8458f8489 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:14.255774 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e93ebf-a55d-456b-bcfd-e5a8458f8489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:14.278395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e93ebf-a55d-456b-bcfd-e5a8458f8489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:14.281411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e93ebf-a55d-456b-bcfd-e5a8458f8489 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:14.294032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1219/2444] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:14.418134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186226f7-8dde-4d3e-877c-a48690344276 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:14.428543 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186226f7-8dde-4d3e-877c-a48690344276 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:14.432298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894fd40-e4ae-46a6-ab16-b5087c7ffcc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:14.433010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894fd40-e4ae-46a6-ab16-b5087c7ffcc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:14.433308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894fd40-e4ae-46a6-ab16-b5087c7ffcc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:14.433598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894fd40-e4ae-46a6-ab16-b5087c7ffcc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:14.503365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894fd40-e4ae-46a6-ab16-b5087c7ffcc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:14.504988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894fd40-e4ae-46a6-ab16-b5087c7ffcc6 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:14.520580 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1226/2445] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:15.293192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e93ebf-a55d-456b-bcfd-e5a8458f8489 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:15.294766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f6e93ebf-a55d-456b-bcfd-e5a8458f8489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:15.306271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e93ebf-a55d-456b-bcfd-e5a8458f8489 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:15.309868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c400c8-8dd0-4c60-b86f-abcec20e1e89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:15.310497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c400c8-8dd0-4c60-b86f-abcec20e1e89 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:15.310777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c400c8-8dd0-4c60-b86f-abcec20e1e89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:15.311051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c400c8-8dd0-4c60-b86f-abcec20e1e89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:15.312721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-39c400c8-8dd0-4c60-b86f-abcec20e1e89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:15.378496 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c400c8-8dd0-4c60-b86f-abcec20e1e89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:15.379945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c400c8-8dd0-4c60-b86f-abcec20e1e89 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:15.389620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1220/2446] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:18.039510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894fd40-e4ae-46a6-ab16-b5087c7ffcc6 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:18.042950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c894fd40-e4ae-46a6-ab16-b5087c7ffcc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:18.131471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894fd40-e4ae-46a6-ab16-b5087c7ffcc6 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:18.136836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:18.137514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:18.137791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:18.138348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] PATH_INFO: `/v3/users/88aecaf73352499aa1e3bda5b06130d6` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:18.140047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=88aecaf73352499aa1e3bda5b06130d6)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:18.142280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:18.225217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88aecaf73352499aa1e3bda5b06130d6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:18.235636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88aecaf73352499aa1e3bda5b06130d6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:18.236706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88aecaf73352499aa1e3bda5b06130d6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:18.239307 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:18.274589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88aecaf73352499aa1e3bda5b06130d6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:18.304950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88aecaf73352499aa1e3bda5b06130d6'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:18.314754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1227/2447] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:18 2022] DELETE /identity/v3/users/88aecaf73352499aa1e3bda5b06130d6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:18.323260 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008157c-0c2a-4d6b-9ab0-fd89cc4abb39 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:18.325234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008157c-0c2a-4d6b-9ab0-fd89cc4abb39 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:18.326853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008157c-0c2a-4d6b-9ab0-fd89cc4abb39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:18.429051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e008157c-0c2a-4d6b-9ab0-fd89cc4abb39 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:18.430787 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e008157c-0c2a-4d6b-9ab0-fd89cc4abb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:18.501197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1221/2448] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:18.511755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:18.514067 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:18.582746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff9d9d-7dbd-44dd-8919-078f7782cc6b admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:18.588161 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa5111-7e46-420d-86a2-9e6a1edf1d5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:18.588273 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa5111-7e46-420d-86a2-9e6a1edf1d5d service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:18.588549 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa5111-7e46-420d-86a2-9e6a1edf1d5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:18.588857 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa5111-7e46-420d-86a2-9e6a1edf1d5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:18.603260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa5111-7e46-420d-86a2-9e6a1edf1d5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:18.604872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa5111-7e46-420d-86a2-9e6a1edf1d5d service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:18.617514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1228/2449] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:18.976939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008157c-0c2a-4d6b-9ab0-fd89cc4abb39 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:18.978754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e008157c-0c2a-4d6b-9ab0-fd89cc4abb39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:18.996765 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008157c-0c2a-4d6b-9ab0-fd89cc4abb39 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:19.015571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:19.016947 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:19.017454 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:19.017932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] PATH_INFO: `/v3/projects/9efb2dff14134f1e9c7d3f9575696b38` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:19.020066 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9efb2dff14134f1e9c7d3f9575696b38)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:19.022893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:19.064889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:19.076524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9efb2dff14134f1e9c7d3f9575696b38', 'request_id': 'req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.083558 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9efb2dff14134f1e9c7d3f9575696b38', 'request_id': 'req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.089494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9efb2dff14134f1e9c7d3f9575696b38', 'request_id': 'req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.101893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] The token cache is being invalidate because project 9efb2dff14134f1e9c7d3f9575696b38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:19.102218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.102884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.104399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1222/2450] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:18 2022] DELETE /identity/v3/projects/9efb2dff14134f1e9c7d3f9575696b38 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:19.109242 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa5111-7e46-420d-86a2-9e6a1edf1d5d service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:19.119950 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa5111-7e46-420d-86a2-9e6a1edf1d5d service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:19.189284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:19.190132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:19.191072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:19.191365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] PATH_INFO: `/v3/users/83099aa35c3d482591c580f0510854f3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:19.194451 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83099aa35c3d482591c580f0510854f3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:19.196238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:19.260482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83099aa35c3d482591c580f0510854f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.271085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83099aa35c3d482591c580f0510854f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.271774 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83099aa35c3d482591c580f0510854f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.282365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83099aa35c3d482591c580f0510854f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.302573 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:19.304707 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83099aa35c3d482591c580f0510854f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.318198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1229/2451] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:19 2022] DELETE /identity/v3/users/83099aa35c3d482591c580f0510854f3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:19.727545 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:19.729096 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:19.746221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79bd7b-5235-4b2f-ae53-51b7cb988f45 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:19.843866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:19.844737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:19.845018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:19.845294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] PATH_INFO: `/v3/projects/52d6d2c1d9424eca8e36194320c24bde` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:19.846607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d6d2c1d9424eca8e36194320c24bde)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:19.848519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:19.933621 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d6d2c1d9424eca8e36194320c24bde', 'request_id': 'req-82e9b837-8306-4bae-af66-d43e77ff2412'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.944180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d6d2c1d9424eca8e36194320c24bde', 'request_id': 'req-82e9b837-8306-4bae-af66-d43e77ff2412'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.955807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d6d2c1d9424eca8e36194320c24bde', 'request_id': 'req-82e9b837-8306-4bae-af66-d43e77ff2412'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.969246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] The token cache is being invalidate because project 52d6d2c1d9424eca8e36194320c24bde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:19.969547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.970267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:19.971996 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1223/2452] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:19 2022] DELETE /identity/v3/projects/52d6d2c1d9424eca8e36194320c24bde => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:19.975949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:19.977374 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:19.992662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad4c139-d199-4e56-a46e-8fd26cde34e6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:20.038428 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:20.040010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:20.067600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb967626-032f-4c5e-9125-b853b8e142a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:20.068248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb967626-032f-4c5e-9125-b853b8e142a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:20.068527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb967626-032f-4c5e-9125-b853b8e142a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:20.068803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb967626-032f-4c5e-9125-b853b8e142a0 admin admin] PATH_INFO: `/v3/users/d1c4658a598c44df9d81542c80914da5` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:20.069924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb967626-032f-4c5e-9125-b853b8e142a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c4658a598c44df9d81542c80914da5)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:20.071884 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb967626-032f-4c5e-9125-b853b8e142a0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:20.097356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:20.112097 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9b837-8306-4bae-af66-d43e77ff2412 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:20.118574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:20.119200 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:20.119478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:20.119759 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] PATH_INFO: `/v3/users/b4d5e044f5b041b7a2404b3ae25bde01` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:20.121065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4d5e044f5b041b7a2404b3ae25bde01)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:20.122120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb967626-032f-4c5e-9125-b853b8e142a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c4658a598c44df9d81542c80914da5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.123988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:20.131283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb967626-032f-4c5e-9125-b853b8e142a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c4658a598c44df9d81542c80914da5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.132176 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb967626-032f-4c5e-9125-b853b8e142a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c4658a598c44df9d81542c80914da5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.141940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb967626-032f-4c5e-9125-b853b8e142a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c4658a598c44df9d81542c80914da5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.158650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb967626-032f-4c5e-9125-b853b8e142a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c4658a598c44df9d81542c80914da5'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.165773 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1230/2453] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:19 2022] DELETE /identity/v3/users/d1c4658a598c44df9d81542c80914da5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:20.181691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4d5e044f5b041b7a2404b3ae25bde01'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.190643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4d5e044f5b041b7a2404b3ae25bde01'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.207422 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4d5e044f5b041b7a2404b3ae25bde01'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.208363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4d5e044f5b041b7a2404b3ae25bde01'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.215757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4d5e044f5b041b7a2404b3ae25bde01'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.222613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1224/2454] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:20 2022] DELETE /identity/v3/users/b4d5e044f5b041b7a2404b3ae25bde01 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:20.227339 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e560c-39aa-4870-9eab-9f509606d003 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:20.227662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e560c-39aa-4870-9eab-9f509606d003 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:20.227972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e560c-39aa-4870-9eab-9f509606d003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:20.251780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b84e560c-39aa-4870-9eab-9f509606d003 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:20.253009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b84e560c-39aa-4870-9eab-9f509606d003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:20.318115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1231/2455] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:20.334159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:20.338654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:20.407307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16b576-8c2d-468f-8e6d-bbfb591d45b5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:20.411789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db61486-9dc3-4fc8-87e6-cb6033b2bb00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:20.412413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db61486-9dc3-4fc8-87e6-cb6033b2bb00 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:20.412700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db61486-9dc3-4fc8-87e6-cb6033b2bb00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:20.412975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db61486-9dc3-4fc8-87e6-cb6033b2bb00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:20.423606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db61486-9dc3-4fc8-87e6-cb6033b2bb00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:20.425056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db61486-9dc3-4fc8-87e6-cb6033b2bb00 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:20.435780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1225/2456] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:20.493592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e560c-39aa-4870-9eab-9f509606d003 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:20.495137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b84e560c-39aa-4870-9eab-9f509606d003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:20.509261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e560c-39aa-4870-9eab-9f509606d003 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:20.525137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:20.525971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:20.526288 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:20.527338 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372 admin admin] PATH_INFO: `/v3/projects/ef4d9846ff5a48d19f7c3010f8bfe832` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:20.543615 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef4d9846ff5a48d19f7c3010f8bfe832)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:20.546904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:20.565147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:20.609779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef4d9846ff5a48d19f7c3010f8bfe832', 'request_id': 'req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.617743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef4d9846ff5a48d19f7c3010f8bfe832', 'request_id': 'req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.629491 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef4d9846ff5a48d19f7c3010f8bfe832', 'request_id': 'req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.645127 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372 admin admin] The token cache is being invalidate because project ef4d9846ff5a48d19f7c3010f8bfe832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:20.645666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.646377 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4d5cae11-e43f-4e2b-b9c6-ca33142f2372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.647919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1232/2457] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:20 2022] DELETE /identity/v3/projects/ef4d9846ff5a48d19f7c3010f8bfe832 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:20.654914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf4528-79ca-4baf-829c-88c9887505f1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:20.655203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf4528-79ca-4baf-829c-88c9887505f1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:20.655481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badf4528-79ca-4baf-829c-88c9887505f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:20.679643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-badf4528-79ca-4baf-829c-88c9887505f1 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:20.680932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-badf4528-79ca-4baf-829c-88c9887505f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:20.748858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1226/2458] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:20.758125 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c37c69-4d60-4474-a395-c59e9077ad75 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:20.758417 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c37c69-4d60-4474-a395-c59e9077ad75 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:20.758774 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c37c69-4d60-4474-a395-c59e9077ad75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:20.770953 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf4528-79ca-4baf-829c-88c9887505f1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:20.773099 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-badf4528-79ca-4baf-829c-88c9887505f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:20.787987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-61c37c69-4d60-4474-a395-c59e9077ad75 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:20.789257 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-61c37c69-4d60-4474-a395-c59e9077ad75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:20.790129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badf4528-79ca-4baf-829c-88c9887505f1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:20.795294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:20.796070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:20.796535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:20.796878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] PATH_INFO: `/v3/projects/3fd26d388dda4c6797e8324b7632bccd` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:20.798235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd26d388dda4c6797e8324b7632bccd)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:20.800146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:20.803931 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1233/2459] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:20.807942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c37c69-4d60-4474-a395-c59e9077ad75 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:20.813204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:20.820597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c37c69-4d60-4474-a395-c59e9077ad75 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:20.827996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22162b1-7f4b-4ad5-8d80-3360cd274b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:20.828701 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22162b1-7f4b-4ad5-8d80-3360cd274b3e admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:20.829049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22162b1-7f4b-4ad5-8d80-3360cd274b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:20.829387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22162b1-7f4b-4ad5-8d80-3360cd274b3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:20.830212 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22162b1-7f4b-4ad5-8d80-3360cd274b3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:20.830588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22162b1-7f4b-4ad5-8d80-3360cd274b3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:20.832354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22162b1-7f4b-4ad5-8d80-3360cd274b3e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:20.856278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1234/2460] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:30:20.857576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22162b1-7f4b-4ad5-8d80-3360cd274b3e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:20.869749 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd26d388dda4c6797e8324b7632bccd', 'request_id': 'req-59319c52-265f-41ef-9788-5b6387b0d214'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.874805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22162b1-7f4b-4ad5-8d80-3360cd274b3e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:20.885840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd26d388dda4c6797e8324b7632bccd', 'request_id': 'req-59319c52-265f-41ef-9788-5b6387b0d214'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.895580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd26d388dda4c6797e8324b7632bccd', 'request_id': 'req-59319c52-265f-41ef-9788-5b6387b0d214'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.910409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] The token cache is being invalidate because project 3fd26d388dda4c6797e8324b7632bccd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:20.910700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.911377 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:20.913098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1227/2461] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:20 2022] DELETE /identity/v3/projects/3fd26d388dda4c6797e8324b7632bccd => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:20.917482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:20.918928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:20.990681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6a487-14ab-4597-8d76-838926c8e010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:20.991521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6a487-14ab-4597-8d76-838926c8e010 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:20.991807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6a487-14ab-4597-8d76-838926c8e010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:20.992084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6a487-14ab-4597-8d76-838926c8e010 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:20.992852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6a487-14ab-4597-8d76-838926c8e010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:20.994852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6a487-14ab-4597-8d76-838926c8e010 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.008430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59319c52-265f-41ef-9788-5b6387b0d214 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:21.014304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:21.015147 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.015515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.015818 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] PATH_INFO: `/v3/users/de9e3a22c33c49b18bfe308c88181a72` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.017726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] RBAC: Authorizing `identity:delete_user(user_id=de9e3a22c33c49b18bfe308c88181a72)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:21.019785 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.027290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1235/2462] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:20 2022] POST /identity/v3/projects => generated 401 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:21.032227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6a487-14ab-4597-8d76-838926c8e010 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:21.034707 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c4b6a487-14ab-4597-8d76-838926c8e010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:21.077127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de9e3a22c33c49b18bfe308c88181a72'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:21.085614 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de9e3a22c33c49b18bfe308c88181a72'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:21.100711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de9e3a22c33c49b18bfe308c88181a72'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:21.101463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de9e3a22c33c49b18bfe308c88181a72'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:21.109539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de9e3a22c33c49b18bfe308c88181a72'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:21.113228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6a487-14ab-4597-8d76-838926c8e010 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:21.115906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1228/2463] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:20 2022] DELETE /identity/v3/users/de9e3a22c33c49b18bfe308c88181a72 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:21.121309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dabc0b-b1e4-4900-aafa-37be83424ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:21.121934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dabc0b-b1e4-4900-aafa-37be83424ccc admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.122245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dabc0b-b1e4-4900-aafa-37be83424ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.122873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dabc0b-b1e4-4900-aafa-37be83424ccc admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.123266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dabc0b-b1e4-4900-aafa-37be83424ccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:21.125221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dabc0b-b1e4-4900-aafa-37be83424ccc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.134956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:21.150240 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1236/2464] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:21 2022] POST /identity/v3/users => generated 301 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:21.155478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:21.156949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:21.179523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:21.233133 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecbdc8-3981-4dbd-b78f-9b261495970d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:21.238466 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd4a83-d9a6-42a0-871d-806522092afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:21.239152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd4a83-d9a6-42a0-871d-806522092afa admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.239448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd4a83-d9a6-42a0-871d-806522092afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.239626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd4a83-d9a6-42a0-871d-806522092afa admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.240375 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd4a83-d9a6-42a0-871d-806522092afa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:21.242082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd4a83-d9a6-42a0-871d-806522092afa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.257397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1229/2465] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:21.258547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dabc0b-b1e4-4900-aafa-37be83424ccc admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:21.288925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dabc0b-b1e4-4900-aafa-37be83424ccc admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:21.297782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0dd13e-0cac-407c-af7f-bf82d5027d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:21.299107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dd13e-0cac-407c-af7f-bf82d5027d48 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.299677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dd13e-0cac-407c-af7f-bf82d5027d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.300213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0dd13e-0cac-407c-af7f-bf82d5027d48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.301654 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dd13e-0cac-407c-af7f-bf82d5027d48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:21.304993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0dd13e-0cac-407c-af7f-bf82d5027d48 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.325063 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1237/2466] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:21.330484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd4a83-d9a6-42a0-871d-806522092afa admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:21.347322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd4a83-d9a6-42a0-871d-806522092afa admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:21.353211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75c004-3f35-4083-907f-aff0b31ac14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:21.354030 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75c004-3f35-4083-907f-aff0b31ac14a admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.355004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75c004-3f35-4083-907f-aff0b31ac14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.355004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75c004-3f35-4083-907f-aff0b31ac14a admin admin] PATH_INFO: `/v3/projects/d20b44ae75a648de8d5bd4bc9fc16a47/users/1d9c7a63791b41ce9ae29b2db335dd6b/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.388514 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75c004-3f35-4083-907f-aff0b31ac14a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d20b44ae75a648de8d5bd4bc9fc16a47, user_id=1d9c7a63791b41ce9ae29b2db335dd6b, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:21.392539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75c004-3f35-4083-907f-aff0b31ac14a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.413432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1230/2467] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:21 2022] PUT /identity/v3/projects/d20b44ae75a648de8d5bd4bc9fc16a47/users/1d9c7a63791b41ce9ae29b2db335dd6b/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:21.424832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0caed-e991-45dd-875c-eb2ac8c22870 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.425150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0caed-e991-45dd-875c-eb2ac8c22870 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.425395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0caed-e991-45dd-875c-eb2ac8c22870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.471482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a1c0caed-e991-45dd-875c-eb2ac8c22870 None None] MFA Rules not processed for user `1d9c7a63791b41ce9ae29b2db335dd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:21.472740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a1c0caed-e991-45dd-875c-eb2ac8c22870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:21.521543 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75c004-3f35-4083-907f-aff0b31ac14a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:21.528178 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9d75c004-3f35-4083-907f-aff0b31ac14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:21.558100 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1238/2468] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:21.564521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0caed-e991-45dd-875c-eb2ac8c22870 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:21.621118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75c004-3f35-4083-907f-aff0b31ac14a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:21.626352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b78fe1-65a9-44cf-af45-267e93116205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:21.627613 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b78fe1-65a9-44cf-af45-267e93116205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.628810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b78fe1-65a9-44cf-af45-267e93116205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.628810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b78fe1-65a9-44cf-af45-267e93116205 admin admin] PATH_INFO: `/v3/projects/937f1c4dfd964cc08a019679f729fefb` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.630340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b78fe1-65a9-44cf-af45-267e93116205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=937f1c4dfd964cc08a019679f729fefb)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:21.632278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b78fe1-65a9-44cf-af45-267e93116205 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.654312 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0caed-e991-45dd-875c-eb2ac8c22870 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:21.660035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f17ce1-d1ad-4bde-ac04-ea7a91a38a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:21.661151 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f17ce1-d1ad-4bde-ac04-ea7a91a38a4c admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.661586 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f17ce1-d1ad-4bde-ac04-ea7a91a38a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.662007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f17ce1-d1ad-4bde-ac04-ea7a91a38a4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.662861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f17ce1-d1ad-4bde-ac04-ea7a91a38a4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:21.665076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f17ce1-d1ad-4bde-ac04-ea7a91a38a4c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.704830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1239/2469] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:21 2022] POST /identity/v3/projects => generated 399 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:21.708964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f17ce1-d1ad-4bde-ac04-ea7a91a38a4c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:21.714767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-90b78fe1-65a9-44cf-af45-267e93116205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '937f1c4dfd964cc08a019679f729fefb', 'request_id': 'req-90b78fe1-65a9-44cf-af45-267e93116205'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:21.731104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-90b78fe1-65a9-44cf-af45-267e93116205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '937f1c4dfd964cc08a019679f729fefb', 'request_id': 'req-90b78fe1-65a9-44cf-af45-267e93116205'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:21.746025 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-90b78fe1-65a9-44cf-af45-267e93116205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '937f1c4dfd964cc08a019679f729fefb', 'request_id': 'req-90b78fe1-65a9-44cf-af45-267e93116205'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:21.762990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-90b78fe1-65a9-44cf-af45-267e93116205 admin admin] The token cache is being invalidate because project 937f1c4dfd964cc08a019679f729fefb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:21.764397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-90b78fe1-65a9-44cf-af45-267e93116205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:21.764397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-90b78fe1-65a9-44cf-af45-267e93116205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:21.765583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1231/2470] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:21 2022] DELETE /identity/v3/projects/937f1c4dfd964cc08a019679f729fefb => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:21.771502 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a95d7-e829-4de9-8ff8-9dce217dab4d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.771747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a95d7-e829-4de9-8ff8-9dce217dab4d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.772020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a95d7-e829-4de9-8ff8-9dce217dab4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.791740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:21.793736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-330a95d7-e829-4de9-8ff8-9dce217dab4d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:21.795585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-330a95d7-e829-4de9-8ff8-9dce217dab4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:21.797052 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f17ce1-d1ad-4bde-ac04-ea7a91a38a4c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:21.804090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098deff-a30c-4eec-9324-fd1c36e69fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:21.805079 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098deff-a30c-4eec-9324-fd1c36e69fe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.805362 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098deff-a30c-4eec-9324-fd1c36e69fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.805636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d098deff-a30c-4eec-9324-fd1c36e69fe1 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.806796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1232/2471] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:21.812991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098deff-a30c-4eec-9324-fd1c36e69fe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:21.815083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d098deff-a30c-4eec-9324-fd1c36e69fe1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.815204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1636b9a-a088-4be8-9ab1-775a16273e03 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.815938 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1636b9a-a088-4be8-9ab1-775a16273e03 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.816347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1636b9a-a088-4be8-9ab1-775a16273e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.838229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f1636b9a-a088-4be8-9ab1-775a16273e03 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:21.842677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1240/2472] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:21 2022] POST /identity/v3/users => generated 306 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:21.842822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f1636b9a-a088-4be8-9ab1-775a16273e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:21.844165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098deff-a30c-4eec-9324-fd1c36e69fe1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:21.847544 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d098deff-a30c-4eec-9324-fd1c36e69fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:21.850869 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1233/2473] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:21.854773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1636b9a-a088-4be8-9ab1-775a16273e03 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:21.863533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d098deff-a30c-4eec-9324-fd1c36e69fe1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:21.865272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1636b9a-a088-4be8-9ab1-775a16273e03 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:21.868381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828f23f-64ea-40a8-a65b-35516f6564eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:21.868986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828f23f-64ea-40a8-a65b-35516f6564eb admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.869278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828f23f-64ea-40a8-a65b-35516f6564eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.869549 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e828f23f-64ea-40a8-a65b-35516f6564eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.870220 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5516e-ca6b-4555-9f95-d5cea43e4a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:21.870810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5516e-ca6b-4555-9f95-d5cea43e4a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.871084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5516e-ca6b-4555-9f95-d5cea43e4a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.871359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c5516e-ca6b-4555-9f95-d5cea43e4a23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.871999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5516e-ca6b-4555-9f95-d5cea43e4a23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:21.872316 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5516e-ca6b-4555-9f95-d5cea43e4a23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:21.873910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c5516e-ca6b-4555-9f95-d5cea43e4a23 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.874804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828f23f-64ea-40a8-a65b-35516f6564eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:21.875741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:21.876780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e828f23f-64ea-40a8-a65b-35516f6564eb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.889245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1241/2474] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:21.893320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1234/2475] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:30:21.894114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828f23f-64ea-40a8-a65b-35516f6564eb admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:21.894286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5516e-ca6b-4555-9f95-d5cea43e4a23 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:21.905203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e828f23f-64ea-40a8-a65b-35516f6564eb admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:21.907543 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c5516e-ca6b-4555-9f95-d5cea43e4a23 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:21.911214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e2c36-fb89-4490-8c84-0c0fd181207f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:21.911983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2c36-fb89-4490-8c84-0c0fd181207f admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.912363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2c36-fb89-4490-8c84-0c0fd181207f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.912702 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2c36-fb89-4490-8c84-0c0fd181207f admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.913611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e2c36-fb89-4490-8c84-0c0fd181207f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:21.915610 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e2c36-fb89-4490-8c84-0c0fd181207f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.916643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56232d74-3fae-4b11-b36d-63c40c3162a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:21.917480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56232d74-3fae-4b11-b36d-63c40c3162a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:21.917792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56232d74-3fae-4b11-b36d-63c40c3162a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:21.918163 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56232d74-3fae-4b11-b36d-63c40c3162a3 admin admin] PATH_INFO: `/v3/projects/8edb85d3cc04408b8d0d2c4df0f76529/users/66705c9019cc49d5b4b58db0a3a402ad/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:21.936642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56232d74-3fae-4b11-b36d-63c40c3162a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8edb85d3cc04408b8d0d2c4df0f76529, user_id=66705c9019cc49d5b4b58db0a3a402ad, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:21.939358 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56232d74-3fae-4b11-b36d-63c40c3162a3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:21.952564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1242/2476] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:21 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:21.956533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e2c36-fb89-4490-8c84-0c0fd181207f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:21.964142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1235/2477] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:21 2022] PUT /identity/v3/projects/8edb85d3cc04408b8d0d2c4df0f76529/users/66705c9019cc49d5b4b58db0a3a402ad/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:21.968658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56232d74-3fae-4b11-b36d-63c40c3162a3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:22.043863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e2c36-fb89-4490-8c84-0c0fd181207f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:22.050945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b9992-d8ad-4aa8-b223-59be28372d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:22.052190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b9992-d8ad-4aa8-b223-59be28372d7c admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:22.052472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b9992-d8ad-4aa8-b223-59be28372d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:22.052759 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b9992-d8ad-4aa8-b223-59be28372d7c admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:22.053395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b9992-d8ad-4aa8-b223-59be28372d7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:22.055538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b9992-d8ad-4aa8-b223-59be28372d7c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:22.062604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56232d74-3fae-4b11-b36d-63c40c3162a3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:22.067919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f883d2-8565-4d12-bfcf-35599d05aa7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:22.068600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f883d2-8565-4d12-bfcf-35599d05aa7b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:22.069248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f883d2-8565-4d12-bfcf-35599d05aa7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:22.069248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f883d2-8565-4d12-bfcf-35599d05aa7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:22.070182 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f883d2-8565-4d12-bfcf-35599d05aa7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:22.071982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f883d2-8565-4d12-bfcf-35599d05aa7b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:22.087387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1236/2478] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:22.087889 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1243/2479] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:21 2022] POST /identity/v3/users => generated 286 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:22.088435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f883d2-8565-4d12-bfcf-35599d05aa7b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:22.089593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b9992-d8ad-4aa8-b223-59be28372d7c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:22.099139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f883d2-8565-4d12-bfcf-35599d05aa7b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:22.100961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b9992-d8ad-4aa8-b223-59be28372d7c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:22.105860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25df0db-a618-4f2f-8618-4afe6e5b89fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:22.106573 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25df0db-a618-4f2f-8618-4afe6e5b89fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:22.106911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25df0db-a618-4f2f-8618-4afe6e5b89fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:22.107092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fdf7f-e496-43ee-ba01-b5034fee4770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:22.107238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25df0db-a618-4f2f-8618-4afe6e5b89fd admin admin] PATH_INFO: `/v3/projects/8edb85d3cc04408b8d0d2c4df0f76529/users/66705c9019cc49d5b4b58db0a3a402ad/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:22.107731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fdf7f-e496-43ee-ba01-b5034fee4770 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:22.108133 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fdf7f-e496-43ee-ba01-b5034fee4770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:22.108479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fdf7f-e496-43ee-ba01-b5034fee4770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:22.109113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25df0db-a618-4f2f-8618-4afe6e5b89fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8edb85d3cc04408b8d0d2c4df0f76529, user_id=66705c9019cc49d5b4b58db0a3a402ad, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:22.109266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fdf7f-e496-43ee-ba01-b5034fee4770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:22.110936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fdf7f-e496-43ee-ba01-b5034fee4770 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:22.111830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25df0db-a618-4f2f-8618-4afe6e5b89fd admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:22.125153 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1237/2480] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:22.125356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1244/2481] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:22 2022] PUT /identity/v3/projects/8edb85d3cc04408b8d0d2c4df0f76529/users/66705c9019cc49d5b4b58db0a3a402ad/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:22.127746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fdf7f-e496-43ee-ba01-b5034fee4770 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:22.136038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756b27d-d76c-40cd-9e57-ee0bfc2e2a89 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:22.136326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756b27d-d76c-40cd-9e57-ee0bfc2e2a89 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:22.136606 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c756b27d-d76c-40cd-9e57-ee0bfc2e2a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:22.141357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fdf7f-e496-43ee-ba01-b5034fee4770 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:22.175235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c756b27d-d76c-40cd-9e57-ee0bfc2e2a89 None None] MFA Rules not processed for user `66705c9019cc49d5b4b58db0a3a402ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:22.176912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c756b27d-d76c-40cd-9e57-ee0bfc2e2a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:22.224496 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442ed69-4c23-4d62-a153-d3964a36aac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:22.225126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442ed69-4c23-4d62-a153-d3964a36aac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:22.225385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442ed69-4c23-4d62-a153-d3964a36aac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:22.225658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442ed69-4c23-4d62-a153-d3964a36aac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:22.226335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442ed69-4c23-4d62-a153-d3964a36aac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:22.228061 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1442ed69-4c23-4d62-a153-d3964a36aac3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:22.238357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1238/2482] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:22 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:22.242880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442ed69-4c23-4d62-a153-d3964a36aac3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:22.254736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442ed69-4c23-4d62-a153-d3964a36aac3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:22.260823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bac19-0edb-451b-9a45-9b1c16b0cae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:22.261562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bac19-0edb-451b-9a45-9b1c16b0cae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:22.261906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bac19-0edb-451b-9a45-9b1c16b0cae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:22.262286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bac19-0edb-451b-9a45-9b1c16b0cae3 admin admin] PATH_INFO: `/v3/projects/661e8005aba1407d8aa0e760fbbaeee0/users/37a46ac5cb774189a4cca5f6880c646c/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:22.280089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bac19-0edb-451b-9a45-9b1c16b0cae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=661e8005aba1407d8aa0e760fbbaeee0, user_id=37a46ac5cb774189a4cca5f6880c646c, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:22.283048 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bac19-0edb-451b-9a45-9b1c16b0cae3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:22.290484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1245/2483] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:22 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:30:22.297535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1239/2484] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:22 2022] PUT /identity/v3/projects/661e8005aba1407d8aa0e760fbbaeee0/users/37a46ac5cb774189a4cca5f6880c646c/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:22.302956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bac19-0edb-451b-9a45-9b1c16b0cae3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:22.303846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34bd0-a32e-473a-9463-92360cb0bf26 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:22.304324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34bd0-a32e-473a-9463-92360cb0bf26 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:22.304665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-307bac19-0edb-451b-9a45-9b1c16b0cae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:22.304799 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb34bd0-a32e-473a-9463-92360cb0bf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:22.325389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-dbb34bd0-a32e-473a-9463-92360cb0bf26 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:22.327204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-dbb34bd0-a32e-473a-9463-92360cb0bf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:22.336815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:22.344843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:22.378162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bac19-0edb-451b-9a45-9b1c16b0cae3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:22.382147 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69b3ae-1040-407f-a16f-ddf90f21e34d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:22.382762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69b3ae-1040-407f-a16f-ddf90f21e34d service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:22.383051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69b3ae-1040-407f-a16f-ddf90f21e34d service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:22.384440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69b3ae-1040-407f-a16f-ddf90f21e34d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:22.404060 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1246/2485] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:22.414360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb34bd0-a32e-473a-9463-92360cb0bf26 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:22.415909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-dbb34bd0-a32e-473a-9463-92360cb0bf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:22.460867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69b3ae-1040-407f-a16f-ddf90f21e34d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:22.462334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69b3ae-1040-407f-a16f-ddf90f21e34d service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:22.474309 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1240/2486] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:22 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:22.482754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb34bd0-a32e-473a-9463-92360cb0bf26 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:22.489574 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6db2614-bfd8-4abb-ace8-6fcd32b87074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:22.490514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db2614-bfd8-4abb-ace8-6fcd32b87074 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:22.490740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db2614-bfd8-4abb-ace8-6fcd32b87074 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:22.490914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6db2614-bfd8-4abb-ace8-6fcd32b87074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:22.501795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db2614-bfd8-4abb-ace8-6fcd32b87074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:22.503282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6db2614-bfd8-4abb-ace8-6fcd32b87074 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:22.520492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1247/2487] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:30:22.569419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69b3ae-1040-407f-a16f-ddf90f21e34d service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:22.579327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69b3ae-1040-407f-a16f-ddf90f21e34d service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:22.583281 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17d02c-7829-4528-86b8-9ed9de9cfa8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:22.583965 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17d02c-7829-4528-86b8-9ed9de9cfa8a service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:22.584304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17d02c-7829-4528-86b8-9ed9de9cfa8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:22.584581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17d02c-7829-4528-86b8-9ed9de9cfa8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:22.587306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-af17d02c-7829-4528-86b8-9ed9de9cfa8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:22.667898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17d02c-7829-4528-86b8-9ed9de9cfa8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:22.669994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17d02c-7829-4528-86b8-9ed9de9cfa8a service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:22.683817 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1241/2488] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:22 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:23.467073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df510c7c-7704-4121-910f-efc6509500da None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:23.467577 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df510c7c-7704-4121-910f-efc6509500da None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:23.467692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df510c7c-7704-4121-910f-efc6509500da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:23.515531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-df510c7c-7704-4121-910f-efc6509500da None None] MFA Rules not processed for user `37a46ac5cb774189a4cca5f6880c646c`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:23.517747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-df510c7c-7704-4121-910f-efc6509500da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:23.612403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1248/2489] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:23.613915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17d02c-7829-4528-86b8-9ed9de9cfa8a service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:23.631090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17d02c-7829-4528-86b8-9ed9de9cfa8a service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:23.639881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7a93b-2b71-46a6-ac12-c3137267d251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:23.640501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7a93b-2b71-46a6-ac12-c3137267d251 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:23.640777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7a93b-2b71-46a6-ac12-c3137267d251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:23.641044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7a93b-2b71-46a6-ac12-c3137267d251 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:23.641946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7a93b-2b71-46a6-ac12-c3137267d251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:23.643960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7a93b-2b71-46a6-ac12-c3137267d251 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:23.706506 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1242/2490] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:23 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:23.706653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df510c7c-7704-4121-910f-efc6509500da None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:23.713762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8bd402-612f-476a-82cb-631804bc705c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:23.714446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8bd402-612f-476a-82cb-631804bc705c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:23.714957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8bd402-612f-476a-82cb-631804bc705c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:23.722318 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df510c7c-7704-4121-910f-efc6509500da None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:23.740570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-da8bd402-612f-476a-82cb-631804bc705c None None] MFA Rules not processed for user `8da8d12ac74644f280ba31da1715f603`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:23.742409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-da8bd402-612f-476a-82cb-631804bc705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:23.796146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=100 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:23.813990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:23.817019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d094569-3db9-4752-bd95-d1531bd2ffd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:23.817871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d094569-3db9-4752-bd95-d1531bd2ffd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:23.818812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d094569-3db9-4752-bd95-d1531bd2ffd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:23.818812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d094569-3db9-4752-bd95-d1531bd2ffd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:23.819503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d094569-3db9-4752-bd95-d1531bd2ffd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:23.821886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d094569-3db9-4752-bd95-d1531bd2ffd3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:23.826555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1243/2491] 10.222.0.64 () {58 vars in 967 bytes} [Wed Sep 21 16:30:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:23.841864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8bd402-612f-476a-82cb-631804bc705c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:23.843890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-da8bd402-612f-476a-82cb-631804bc705c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:23.851181 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1249/2492] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:23 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:23.853787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d094569-3db9-4752-bd95-d1531bd2ffd3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:23.866140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d094569-3db9-4752-bd95-d1531bd2ffd3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:23.873742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e062ee-810a-49cb-88a4-a4603f617f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:23.874506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e062ee-810a-49cb-88a4-a4603f617f24 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:23.875201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e062ee-810a-49cb-88a4-a4603f617f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:23.875486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e062ee-810a-49cb-88a4-a4603f617f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:23.876435 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e062ee-810a-49cb-88a4-a4603f617f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:23.878076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e062ee-810a-49cb-88a4-a4603f617f24 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:23.894092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1250/2493] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:23.896238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e062ee-810a-49cb-88a4-a4603f617f24 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:23.911464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e062ee-810a-49cb-88a4-a4603f617f24 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:23.917782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bb081-5a50-40d8-8d5e-dc4d07f70659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:23.918752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bb081-5a50-40d8-8d5e-dc4d07f70659 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:23.919065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bb081-5a50-40d8-8d5e-dc4d07f70659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:23.920198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32bb081-5a50-40d8-8d5e-dc4d07f70659 admin admin] PATH_INFO: `/v3/projects/b40dd686c68d462fbe7962e1ecb71c1a/users/437ea6e388fd47d78f9d9935ae4c5cfe/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:23.936110 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bb081-5a50-40d8-8d5e-dc4d07f70659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b40dd686c68d462fbe7962e1ecb71c1a, user_id=437ea6e388fd47d78f9d9935ae4c5cfe, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:23.937851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8bd402-612f-476a-82cb-631804bc705c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:23.938970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32bb081-5a50-40d8-8d5e-dc4d07f70659 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:23.943196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc1632-d142-4ca6-b59c-726f4a423933 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a1b8081ce490db46db95f14024ba6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:23.943817 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc1632-d142-4ca6-b59c-726f4a423933 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:23.944095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc1632-d142-4ca6-b59c-726f4a423933 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:23.944424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc1632-d142-4ca6-b59c-726f4a423933 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:23.954134 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc1632-d142-4ca6-b59c-726f4a423933 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:23.955727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc1632-d142-4ca6-b59c-726f4a423933 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:23.956903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1251/2494] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:23 2022] PUT /identity/v3/projects/b40dd686c68d462fbe7962e1ecb71c1a/users/437ea6e388fd47d78f9d9935ae4c5cfe/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:23.960696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bb081-5a50-40d8-8d5e-dc4d07f70659 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:23.967046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1244/2495] 10.222.0.64 () {60 vars in 1384 bytes} [Wed Sep 21 16:30:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 21 16:30:24.043794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32bb081-5a50-40d8-8d5e-dc4d07f70659 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:24.049184 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e118de-2934-4dd8-8eb1-af17ce0881bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:24.049806 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e118de-2934-4dd8-8eb1-af17ce0881bd admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:24.050109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e118de-2934-4dd8-8eb1-af17ce0881bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:24.050876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e118de-2934-4dd8-8eb1-af17ce0881bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:24.051020 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e118de-2934-4dd8-8eb1-af17ce0881bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:24.052823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e118de-2934-4dd8-8eb1-af17ce0881bd admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:24.070614 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1252/2496] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:23 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:24.071778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc1632-d142-4ca6-b59c-726f4a423933 service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:24.083433 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc1632-d142-4ca6-b59c-726f4a423933 service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:24.088839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1111b-df66-4d43-b1f6-484eb09be390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:24.089430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1111b-df66-4d43-b1f6-484eb09be390 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:24.089693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1111b-df66-4d43-b1f6-484eb09be390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:24.089962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1111b-df66-4d43-b1f6-484eb09be390 admin admin] PATH_INFO: `/v3/projects/b40dd686c68d462fbe7962e1ecb71c1a/users/437ea6e388fd47d78f9d9935ae4c5cfe/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:24.092702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1111b-df66-4d43-b1f6-484eb09be390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b40dd686c68d462fbe7962e1ecb71c1a, user_id=437ea6e388fd47d78f9d9935ae4c5cfe, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:24.095494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1111b-df66-4d43-b1f6-484eb09be390 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:24.107629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1245/2497] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:24 2022] PUT /identity/v3/projects/b40dd686c68d462fbe7962e1ecb71c1a/users/437ea6e388fd47d78f9d9935ae4c5cfe/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:25.363260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c48514-867a-4382-97c1-8b0ade0f118c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:25.364117 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c48514-867a-4382-97c1-8b0ade0f118c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:25.364420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c48514-867a-4382-97c1-8b0ade0f118c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:25.404283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-58c48514-867a-4382-97c1-8b0ade0f118c None None] MFA Rules not processed for user `437ea6e388fd47d78f9d9935ae4c5cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:25.406034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-58c48514-867a-4382-97c1-8b0ade0f118c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:25.543126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1253/2498] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:25.555097 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1111b-df66-4d43-b1f6-484eb09be390 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:25.618989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1111b-df66-4d43-b1f6-484eb09be390 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:25.622712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6bebb-e25e-4239-b858-5e489503cc75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:25.623324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6bebb-e25e-4239-b858-5e489503cc75 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:25.623604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6bebb-e25e-4239-b858-5e489503cc75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:25.623881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6bebb-e25e-4239-b858-5e489503cc75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:25.699667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6bebb-e25e-4239-b858-5e489503cc75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:25.701832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6bebb-e25e-4239-b858-5e489503cc75 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:25.747781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1246/2499] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:27.246760 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c48514-867a-4382-97c1-8b0ade0f118c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:27.248306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-58c48514-867a-4382-97c1-8b0ade0f118c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:27.260613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c48514-867a-4382-97c1-8b0ade0f118c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:27.264989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24478d-bc5b-4a68-aedc-fda6e4f2fcaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:27.265638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24478d-bc5b-4a68-aedc-fda6e4f2fcaf service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:27.265913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24478d-bc5b-4a68-aedc-fda6e4f2fcaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:27.266229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24478d-bc5b-4a68-aedc-fda6e4f2fcaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:27.277612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24478d-bc5b-4a68-aedc-fda6e4f2fcaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:27.279889 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24478d-bc5b-4a68-aedc-fda6e4f2fcaf service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:27.289804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1254/2500] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:27.457272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6bebb-e25e-4239-b858-5e489503cc75 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:27.496633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:27.548175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6bebb-e25e-4239-b858-5e489503cc75 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:27.552865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:27.553492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:27.553767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:27.554037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] PATH_INFO: `/v3/users/1d9c7a63791b41ce9ae29b2db335dd6b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:27.555178 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d9c7a63791b41ce9ae29b2db335dd6b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:27.557067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:27.611276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d9c7a63791b41ce9ae29b2db335dd6b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:27.619644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d9c7a63791b41ce9ae29b2db335dd6b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:27.632594 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d9c7a63791b41ce9ae29b2db335dd6b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:27.633349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d9c7a63791b41ce9ae29b2db335dd6b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:27.641706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d9c7a63791b41ce9ae29b2db335dd6b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:27.648432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1247/2501] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:27 2022] DELETE /identity/v3/users/1d9c7a63791b41ce9ae29b2db335dd6b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:27.652987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38f8bd-58ac-40d5-9f97-afee88746b7f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:27.653816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38f8bd-58ac-40d5-9f97-afee88746b7f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:27.654121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38f8bd-58ac-40d5-9f97-afee88746b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:27.681352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6f38f8bd-58ac-40d5-9f97-afee88746b7f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:27.683031 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6f38f8bd-58ac-40d5-9f97-afee88746b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:27.690412 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:27.755522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1255/2502] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:27.765547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:27.767337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:27.828149 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198551-f471-4c22-996f-a3f57b593f14 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:27.831821 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b39b72-c9d3-4f91-ac24-9995a47286d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:27.832423 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b39b72-c9d3-4f91-ac24-9995a47286d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:27.832706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b39b72-c9d3-4f91-ac24-9995a47286d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:27.832980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b39b72-c9d3-4f91-ac24-9995a47286d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:27.842564 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b39b72-c9d3-4f91-ac24-9995a47286d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:27.843974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b39b72-c9d3-4f91-ac24-9995a47286d9 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:27.853305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1248/2503] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:28.165553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38f8bd-58ac-40d5-9f97-afee88746b7f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:28.167078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6f38f8bd-58ac-40d5-9f97-afee88746b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:28.207917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38f8bd-58ac-40d5-9f97-afee88746b7f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:28.214469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:28.215287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:28.216697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:28.217001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] PATH_INFO: `/v3/projects/d20b44ae75a648de8d5bd4bc9fc16a47` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:28.219076 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d20b44ae75a648de8d5bd4bc9fc16a47)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:28.221319 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:28.280283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd20b44ae75a648de8d5bd4bc9fc16a47', 'request_id': 'req-eb1a4001-0bea-422f-9994-169a79004f2e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:28.288198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd20b44ae75a648de8d5bd4bc9fc16a47', 'request_id': 'req-eb1a4001-0bea-422f-9994-169a79004f2e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:28.295658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd20b44ae75a648de8d5bd4bc9fc16a47', 'request_id': 'req-eb1a4001-0bea-422f-9994-169a79004f2e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:28.306818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] The token cache is being invalidate because project d20b44ae75a648de8d5bd4bc9fc16a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:28.307113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:28.307972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:28.309517 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1256/2504] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:28 2022] DELETE /identity/v3/projects/d20b44ae75a648de8d5bd4bc9fc16a47 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:28.313598 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b39b72-c9d3-4f91-ac24-9995a47286d9 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:28.329050 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b39b72-c9d3-4f91-ac24-9995a47286d9 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:28.384175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:28.399623 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c28123-624d-4838-b959-b0532c19995a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:28.399623 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c28123-624d-4838-b959-b0532c19995a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:28.399623 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c28123-624d-4838-b959-b0532c19995a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:28.399623 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c28123-624d-4838-b959-b0532c19995a admin admin] PATH_INFO: `/v3/users/66705c9019cc49d5b4b58db0a3a402ad` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:28.401706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c28123-624d-4838-b959-b0532c19995a admin admin] RBAC: Authorizing `identity:delete_user(user_id=66705c9019cc49d5b4b58db0a3a402ad)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:28.406483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c28123-624d-4838-b959-b0532c19995a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:28.469585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f4c28123-624d-4838-b959-b0532c19995a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66705c9019cc49d5b4b58db0a3a402ad'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:28.485430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f4c28123-624d-4838-b959-b0532c19995a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66705c9019cc49d5b4b58db0a3a402ad'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:28.501376 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f4c28123-624d-4838-b959-b0532c19995a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66705c9019cc49d5b4b58db0a3a402ad'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:28.502158 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f4c28123-624d-4838-b959-b0532c19995a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66705c9019cc49d5b4b58db0a3a402ad'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:28.512118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f4c28123-624d-4838-b959-b0532c19995a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66705c9019cc49d5b4b58db0a3a402ad'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:28.521217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1249/2505] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:28 2022] DELETE /identity/v3/users/66705c9019cc49d5b4b58db0a3a402ad => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:29.392828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:29.394031 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:29.422683 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1a4001-0bea-422f-9994-169a79004f2e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:29.511144 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a426d95-ca5f-4dac-b402-aad12104c422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:29.514635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a426d95-ca5f-4dac-b402-aad12104c422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:29.515766 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a426d95-ca5f-4dac-b402-aad12104c422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:29.516590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a426d95-ca5f-4dac-b402-aad12104c422 admin admin] PATH_INFO: `/v3/projects/8edb85d3cc04408b8d0d2c4df0f76529` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:29.518445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a426d95-ca5f-4dac-b402-aad12104c422 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8edb85d3cc04408b8d0d2c4df0f76529)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:29.520549 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a426d95-ca5f-4dac-b402-aad12104c422 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:29.563458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:29.582704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4a426d95-ca5f-4dac-b402-aad12104c422 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8edb85d3cc04408b8d0d2c4df0f76529', 'request_id': 'req-4a426d95-ca5f-4dac-b402-aad12104c422'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:29.593711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4a426d95-ca5f-4dac-b402-aad12104c422 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8edb85d3cc04408b8d0d2c4df0f76529', 'request_id': 'req-4a426d95-ca5f-4dac-b402-aad12104c422'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:29.601671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4a426d95-ca5f-4dac-b402-aad12104c422 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8edb85d3cc04408b8d0d2c4df0f76529', 'request_id': 'req-4a426d95-ca5f-4dac-b402-aad12104c422'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:29.613331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4a426d95-ca5f-4dac-b402-aad12104c422 admin admin] The token cache is being invalidate because project 8edb85d3cc04408b8d0d2c4df0f76529 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:29.613623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4a426d95-ca5f-4dac-b402-aad12104c422 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:29.614321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4a426d95-ca5f-4dac-b402-aad12104c422 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:29.616210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1257/2506] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:29 2022] DELETE /identity/v3/projects/8edb85d3cc04408b8d0d2c4df0f76529 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:29.621351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443042d2-1530-4615-88dd-a2cce93a24cf None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:29.624063 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443042d2-1530-4615-88dd-a2cce93a24cf None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:29.624063 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443042d2-1530-4615-88dd-a2cce93a24cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:29.648936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-443042d2-1530-4615-88dd-a2cce93a24cf None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:29.650319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-443042d2-1530-4615-88dd-a2cce93a24cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:29.745919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1250/2507] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:29.753997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a5b5f-ad86-493c-915a-96e948ddf61b None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:29.754458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a5b5f-ad86-493c-915a-96e948ddf61b None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:29.754848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11a5b5f-ad86-493c-915a-96e948ddf61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:29.777717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e11a5b5f-ad86-493c-915a-96e948ddf61b None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:29.780178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e11a5b5f-ad86-493c-915a-96e948ddf61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:29.790100 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1258/2508] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:29.793795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443042d2-1530-4615-88dd-a2cce93a24cf None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:29.803894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443042d2-1530-4615-88dd-a2cce93a24cf None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:29.808968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99a182-0736-4464-8ddc-4910a2cf2df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:29.810183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99a182-0736-4464-8ddc-4910a2cf2df6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:29.810472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99a182-0736-4464-8ddc-4910a2cf2df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:29.810757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99a182-0736-4464-8ddc-4910a2cf2df6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:29.811432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99a182-0736-4464-8ddc-4910a2cf2df6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:29.811737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99a182-0736-4464-8ddc-4910a2cf2df6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:29.813424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99a182-0736-4464-8ddc-4910a2cf2df6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:29.836283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1251/2509] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:30:29.839826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a5b5f-ad86-493c-915a-96e948ddf61b None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:29.852406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11a5b5f-ad86-493c-915a-96e948ddf61b None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:29.859538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8b9b7-9f10-477f-a9aa-7eae02b07ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:29.860316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b9b7-9f10-477f-a9aa-7eae02b07ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:29.860508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b9b7-9f10-477f-a9aa-7eae02b07ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:29.860852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8b9b7-9f10-477f-a9aa-7eae02b07ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:29.861410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8b9b7-9f10-477f-a9aa-7eae02b07ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:29.863594 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8b9b7-9f10-477f-a9aa-7eae02b07ce1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:29.902347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1259/2510] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:29 2022] POST /identity/v3/projects => generated 389 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:29.907243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99a182-0736-4464-8ddc-4910a2cf2df6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:29.917478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99a182-0736-4464-8ddc-4910a2cf2df6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:29.975518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:29.983276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6df63-cdc1-4db1-9f00-35e34aad28f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:29.984312 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6df63-cdc1-4db1-9f00-35e34aad28f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:29.984619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6df63-cdc1-4db1-9f00-35e34aad28f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:29.984864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6df63-cdc1-4db1-9f00-35e34aad28f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:29.985624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6df63-cdc1-4db1-9f00-35e34aad28f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:29.987994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6df63-cdc1-4db1-9f00-35e34aad28f8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:30.015257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1252/2511] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:29 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:30.019921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8b9b7-9f10-477f-a9aa-7eae02b07ce1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:30.031815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8b9b7-9f10-477f-a9aa-7eae02b07ce1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:30.036813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4364e9-8666-4e49-bbec-f9cd2a6215b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:30.037463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4364e9-8666-4e49-bbec-f9cd2a6215b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:30.037867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4364e9-8666-4e49-bbec-f9cd2a6215b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:30.038192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4364e9-8666-4e49-bbec-f9cd2a6215b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:30.038859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4364e9-8666-4e49-bbec-f9cd2a6215b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:30.040616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4364e9-8666-4e49-bbec-f9cd2a6215b6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:30.059198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1260/2512] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:30.061151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6df63-cdc1-4db1-9f00-35e34aad28f8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:30.071215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6df63-cdc1-4db1-9f00-35e34aad28f8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:30.076919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032c38d-5e0b-44ee-82d0-305d8da05e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:30.077536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032c38d-5e0b-44ee-82d0-305d8da05e58 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:30.077822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032c38d-5e0b-44ee-82d0-305d8da05e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:30.078125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032c38d-5e0b-44ee-82d0-305d8da05e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:30.079148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032c38d-5e0b-44ee-82d0-305d8da05e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:30.080776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032c38d-5e0b-44ee-82d0-305d8da05e58 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:30.094667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1253/2513] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:30.098996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4364e9-8666-4e49-bbec-f9cd2a6215b6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:30.111739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4364e9-8666-4e49-bbec-f9cd2a6215b6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:30.117803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443f11e-d7da-42cc-a7ae-802e127ad27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:30.118660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443f11e-d7da-42cc-a7ae-802e127ad27f admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:30.119046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443f11e-d7da-42cc-a7ae-802e127ad27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:30.119423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443f11e-d7da-42cc-a7ae-802e127ad27f admin admin] PATH_INFO: `/v3/projects/a9311d932b2f44318d2ddf8ae393c0fb/users/67ce12a57dd34355ba2613ecd1462304/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:30.143426 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443f11e-d7da-42cc-a7ae-802e127ad27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9311d932b2f44318d2ddf8ae393c0fb, user_id=67ce12a57dd34355ba2613ecd1462304, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:30.146245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6443f11e-d7da-42cc-a7ae-802e127ad27f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:30.171120 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1261/2514] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:30 2022] PUT /identity/v3/projects/a9311d932b2f44318d2ddf8ae393c0fb/users/67ce12a57dd34355ba2613ecd1462304/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:30.181778 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f736f0-86f9-4981-99b2-c91a4588aa2f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:30.182405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f736f0-86f9-4981-99b2-c91a4588aa2f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:30.183072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f736f0-86f9-4981-99b2-c91a4588aa2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:30.233045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-02f736f0-86f9-4981-99b2-c91a4588aa2f None None] MFA Rules not processed for user `67ce12a57dd34355ba2613ecd1462304`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:30.234675 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-02f736f0-86f9-4981-99b2-c91a4588aa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:30.352405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1254/2515] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:30.357687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443f11e-d7da-42cc-a7ae-802e127ad27f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:30.451107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6443f11e-d7da-42cc-a7ae-802e127ad27f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:30.460640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba125d-b35e-4470-9623-340ec5de34b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:30.461685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba125d-b35e-4470-9623-340ec5de34b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:30.462192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba125d-b35e-4470-9623-340ec5de34b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:30.462688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba125d-b35e-4470-9623-340ec5de34b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:30.463516 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba125d-b35e-4470-9623-340ec5de34b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:30.466879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba125d-b35e-4470-9623-340ec5de34b2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:30.528980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f736f0-86f9-4981-99b2-c91a4588aa2f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:30.529748 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1262/2516] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:30 2022] POST /identity/v3/projects => generated 389 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:30.544607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f736f0-86f9-4981-99b2-c91a4588aa2f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:30.628530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49e71f-ea30-4283-a8a7-e74a3266419f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:30.629238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49e71f-ea30-4283-a8a7-e74a3266419f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:30.629338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49e71f-ea30-4283-a8a7-e74a3266419f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:30.629590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49e71f-ea30-4283-a8a7-e74a3266419f admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:30.630289 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49e71f-ea30-4283-a8a7-e74a3266419f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:30.632224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49e71f-ea30-4283-a8a7-e74a3266419f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:30.675731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1255/2517] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:30 2022] POST /identity/v3/users => generated 301 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:30.681022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba125d-b35e-4470-9623-340ec5de34b2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:30.700705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba125d-b35e-4470-9623-340ec5de34b2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:30.714463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663061ac-93cb-4fd1-a303-92111cfd8aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:30.715964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663061ac-93cb-4fd1-a303-92111cfd8aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:30.716258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663061ac-93cb-4fd1-a303-92111cfd8aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:30.716533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663061ac-93cb-4fd1-a303-92111cfd8aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:30.717200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663061ac-93cb-4fd1-a303-92111cfd8aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:30.718868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663061ac-93cb-4fd1-a303-92111cfd8aa0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:30.736625 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1263/2518] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:30.737914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49e71f-ea30-4283-a8a7-e74a3266419f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:30.748642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49e71f-ea30-4283-a8a7-e74a3266419f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:30.756377 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15ac6c-43de-426a-ab82-b01101ea3b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:30.757217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15ac6c-43de-426a-ab82-b01101ea3b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:30.757565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15ac6c-43de-426a-ab82-b01101ea3b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:30.758107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15ac6c-43de-426a-ab82-b01101ea3b03 admin admin] PATH_INFO: `/v3/projects/4a284d0a6199424d96bc2bd9da05810b/users/1b8fe1634808432184b8803c349af992/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:30.780817 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15ac6c-43de-426a-ab82-b01101ea3b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a284d0a6199424d96bc2bd9da05810b, user_id=1b8fe1634808432184b8803c349af992, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:30.784250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15ac6c-43de-426a-ab82-b01101ea3b03 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:30.807884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1256/2519] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:30 2022] PUT /identity/v3/projects/4a284d0a6199424d96bc2bd9da05810b/users/1b8fe1634808432184b8803c349af992/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:30.815145 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663061ac-93cb-4fd1-a303-92111cfd8aa0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:30.830283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663061ac-93cb-4fd1-a303-92111cfd8aa0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:30.886765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:30.938839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2870e-022d-40d5-94c8-5319415ad48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:30.939680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2870e-022d-40d5-94c8-5319415ad48a admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:30.940105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2870e-022d-40d5-94c8-5319415ad48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:30.940606 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2870e-022d-40d5-94c8-5319415ad48a admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:30.941569 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2870e-022d-40d5-94c8-5319415ad48a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:30.943775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c2870e-022d-40d5-94c8-5319415ad48a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:30.964547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1264/2520] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:30.966835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15ac6c-43de-426a-ab82-b01101ea3b03 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:30.980307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15ac6c-43de-426a-ab82-b01101ea3b03 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:30.989933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fc568-32d9-4aa3-a031-314e5cee80b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:30.991456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fc568-32d9-4aa3-a031-314e5cee80b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:30.991747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fc568-32d9-4aa3-a031-314e5cee80b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:30.992055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fc568-32d9-4aa3-a031-314e5cee80b9 admin admin] PATH_INFO: `/v3/projects/4a284d0a6199424d96bc2bd9da05810b/users/1b8fe1634808432184b8803c349af992/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:30.994207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fc568-32d9-4aa3-a031-314e5cee80b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a284d0a6199424d96bc2bd9da05810b, user_id=1b8fe1634808432184b8803c349af992, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:30.996970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fc568-32d9-4aa3-a031-314e5cee80b9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:31.013215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1257/2521] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:30 2022] PUT /identity/v3/projects/4a284d0a6199424d96bc2bd9da05810b/users/1b8fe1634808432184b8803c349af992/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:31.022947 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b87c88-20c6-40f3-9a86-d0a206b5c85c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:31.023306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b87c88-20c6-40f3-9a86-d0a206b5c85c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:31.023607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b87c88-20c6-40f3-9a86-d0a206b5c85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:31.057344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f5b87c88-20c6-40f3-9a86-d0a206b5c85c None None] MFA Rules not processed for user `1b8fe1634808432184b8803c349af992`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:31.059498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f5b87c88-20c6-40f3-9a86-d0a206b5c85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:31.160135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1265/2522] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:31 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:31.186507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fc568-32d9-4aa3-a031-314e5cee80b9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:31.188498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-552fc568-32d9-4aa3-a031-314e5cee80b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:31.219279 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:31.282732 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fc568-32d9-4aa3-a031-314e5cee80b9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:31.288252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34333c84-a4a2-457f-9651-d978f3cfe078 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:31.289084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34333c84-a4a2-457f-9651-d978f3cfe078 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:31.289589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34333c84-a4a2-457f-9651-d978f3cfe078 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:31.290040 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34333c84-a4a2-457f-9651-d978f3cfe078 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:31.302277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34333c84-a4a2-457f-9651-d978f3cfe078 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:31.303752 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34333c84-a4a2-457f-9651-d978f3cfe078 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:31.314299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1258/2523] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:31 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:31.495109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b87c88-20c6-40f3-9a86-d0a206b5c85c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:31.625043 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b87c88-20c6-40f3-9a86-d0a206b5c85c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:31.630679 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:31.631420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:31.631702 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:31.632010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] PATH_INFO: `/v3/users/67ce12a57dd34355ba2613ecd1462304` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:31.633277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67ce12a57dd34355ba2613ecd1462304)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:31.635277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:31.709167 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67ce12a57dd34355ba2613ecd1462304'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:31.726302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67ce12a57dd34355ba2613ecd1462304'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:31.726302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67ce12a57dd34355ba2613ecd1462304'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:31.730628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:31.746293 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67ce12a57dd34355ba2613ecd1462304'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:31.775995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67ce12a57dd34355ba2613ecd1462304'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:31.788779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1266/2524] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:31 2022] DELETE /identity/v3/users/67ce12a57dd34355ba2613ecd1462304 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:31.793239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df162c08-6bb4-48d9-ab50-6384b521dbf5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:31.794203 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df162c08-6bb4-48d9-ab50-6384b521dbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:31.794647 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df162c08-6bb4-48d9-ab50-6384b521dbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:31.825568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-df162c08-6bb4-48d9-ab50-6384b521dbf5 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:31.826916 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-df162c08-6bb4-48d9-ab50-6384b521dbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:31.911192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1259/2525] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:31.921072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:31.922704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:31.996295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e02e54-233d-4134-a13c-63e258d21004 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:32.001871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa64730-3e33-4c56-a41d-d44e88287037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:32.002838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa64730-3e33-4c56-a41d-d44e88287037 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:32.003268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa64730-3e33-4c56-a41d-d44e88287037 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:32.003743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa64730-3e33-4c56-a41d-d44e88287037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:32.016635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa64730-3e33-4c56-a41d-d44e88287037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:32.018678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa64730-3e33-4c56-a41d-d44e88287037 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:32.032478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1267/2526] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:32.647214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df162c08-6bb4-48d9-ab50-6384b521dbf5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:32.649763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-df162c08-6bb4-48d9-ab50-6384b521dbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:32.672196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df162c08-6bb4-48d9-ab50-6384b521dbf5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:32.681405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:32.682369 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:32.682816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:32.687177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] PATH_INFO: `/v3/projects/a9311d932b2f44318d2ddf8ae393c0fb` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:32.688570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9311d932b2f44318d2ddf8ae393c0fb)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:32.690931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:32.753566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9311d932b2f44318d2ddf8ae393c0fb', 'request_id': 'req-571738e3-10c6-48b5-b0fc-2e1f1640f91a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:32.761211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9311d932b2f44318d2ddf8ae393c0fb', 'request_id': 'req-571738e3-10c6-48b5-b0fc-2e1f1640f91a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:32.769587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9311d932b2f44318d2ddf8ae393c0fb', 'request_id': 'req-571738e3-10c6-48b5-b0fc-2e1f1640f91a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:32.783946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] The token cache is being invalidate because project a9311d932b2f44318d2ddf8ae393c0fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:32.784243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:32.784971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:32.786625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1260/2527] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:32 2022] DELETE /identity/v3/projects/a9311d932b2f44318d2ddf8ae393c0fb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:32.790976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa64730-3e33-4c56-a41d-d44e88287037 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:32.803586 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa64730-3e33-4c56-a41d-d44e88287037 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:32.875500 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fedddd-fd81-4603-bcea-528bf664e8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:32.876253 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fedddd-fd81-4603-bcea-528bf664e8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:32.876717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fedddd-fd81-4603-bcea-528bf664e8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:32.877129 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fedddd-fd81-4603-bcea-528bf664e8c6 admin admin] PATH_INFO: `/v3/users/1b8fe1634808432184b8803c349af992` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:32.879379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fedddd-fd81-4603-bcea-528bf664e8c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8fe1634808432184b8803c349af992)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:32.881442 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fedddd-fd81-4603-bcea-528bf664e8c6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:32.948546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-02fedddd-fd81-4603-bcea-528bf664e8c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8fe1634808432184b8803c349af992'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:32.958284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-02fedddd-fd81-4603-bcea-528bf664e8c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8fe1634808432184b8803c349af992'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:32.959479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-02fedddd-fd81-4603-bcea-528bf664e8c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8fe1634808432184b8803c349af992'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:32.969223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-02fedddd-fd81-4603-bcea-528bf664e8c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8fe1634808432184b8803c349af992'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:32.987546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:32.989366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-02fedddd-fd81-4603-bcea-528bf664e8c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8fe1634808432184b8803c349af992'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:33.000100 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1268/2528] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:32 2022] DELETE /identity/v3/users/1b8fe1634808432184b8803c349af992 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:33.351665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:33.353685 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:33.371346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571738e3-10c6-48b5-b0fc-2e1f1640f91a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:33.428617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:33.449154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea40da50-a53c-4133-a271-79421f76caf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:33.450498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40da50-a53c-4133-a271-79421f76caf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:33.450781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40da50-a53c-4133-a271-79421f76caf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:33.451056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea40da50-a53c-4133-a271-79421f76caf2 admin admin] PATH_INFO: `/v3/projects/4a284d0a6199424d96bc2bd9da05810b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:33.452264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40da50-a53c-4133-a271-79421f76caf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a284d0a6199424d96bc2bd9da05810b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:33.454215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea40da50-a53c-4133-a271-79421f76caf2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:33.519289 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ea40da50-a53c-4133-a271-79421f76caf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a284d0a6199424d96bc2bd9da05810b', 'request_id': 'req-ea40da50-a53c-4133-a271-79421f76caf2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:33.529771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ea40da50-a53c-4133-a271-79421f76caf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a284d0a6199424d96bc2bd9da05810b', 'request_id': 'req-ea40da50-a53c-4133-a271-79421f76caf2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:33.536479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ea40da50-a53c-4133-a271-79421f76caf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a284d0a6199424d96bc2bd9da05810b', 'request_id': 'req-ea40da50-a53c-4133-a271-79421f76caf2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:33.550555 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ea40da50-a53c-4133-a271-79421f76caf2 admin admin] The token cache is being invalidate because project 4a284d0a6199424d96bc2bd9da05810b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:33.550875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ea40da50-a53c-4133-a271-79421f76caf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:33.551751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ea40da50-a53c-4133-a271-79421f76caf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:33.553261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1261/2529] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:33 2022] DELETE /identity/v3/projects/4a284d0a6199424d96bc2bd9da05810b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:33.558407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46e532-391a-4918-9e94-565c2aa8b1da None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:33.558699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46e532-391a-4918-9e94-565c2aa8b1da None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:33.559692 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46e532-391a-4918-9e94-565c2aa8b1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:33.591270 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0c46e532-391a-4918-9e94-565c2aa8b1da None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:33.591270 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0c46e532-391a-4918-9e94-565c2aa8b1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:33.672130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1269/2530] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:33.684300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628fc58-6c0f-4c77-9984-7c09ebf4555c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:33.684587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628fc58-6c0f-4c77-9984-7c09ebf4555c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:33.684864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628fc58-6c0f-4c77-9984-7c09ebf4555c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:33.710577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f628fc58-6c0f-4c77-9984-7c09ebf4555c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:33.711775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f628fc58-6c0f-4c77-9984-7c09ebf4555c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:33.730126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1262/2531] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:33.732824 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46e532-391a-4918-9e94-565c2aa8b1da None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:33.748967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46e532-391a-4918-9e94-565c2aa8b1da None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:33.754529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51960fe-c22f-4499-aa2d-5c40d6d72461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:33.755183 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51960fe-c22f-4499-aa2d-5c40d6d72461 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:33.755492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51960fe-c22f-4499-aa2d-5c40d6d72461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:33.755740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51960fe-c22f-4499-aa2d-5c40d6d72461 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:33.756399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51960fe-c22f-4499-aa2d-5c40d6d72461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:33.756700 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51960fe-c22f-4499-aa2d-5c40d6d72461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:33.758349 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51960fe-c22f-4499-aa2d-5c40d6d72461 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:33.776248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1270/2532] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:30:33.781488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628fc58-6c0f-4c77-9984-7c09ebf4555c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:33.791943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628fc58-6c0f-4c77-9984-7c09ebf4555c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:33.798984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b4f97-fc61-4332-992c-c45b4abd83fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:33.799607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b4f97-fc61-4332-992c-c45b4abd83fd admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:33.799885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b4f97-fc61-4332-992c-c45b4abd83fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:33.800156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b4f97-fc61-4332-992c-c45b4abd83fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:33.800805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b4f97-fc61-4332-992c-c45b4abd83fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:33.802753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b4f97-fc61-4332-992c-c45b4abd83fd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:33.838517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1263/2533] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:33 2022] POST /identity/v3/projects => generated 409 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:33.843699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51960fe-c22f-4499-aa2d-5c40d6d72461 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:33.860346 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51960fe-c22f-4499-aa2d-5c40d6d72461 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:33.952895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa1d86-797e-45d2-967e-f36d6959f28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:33.953482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa1d86-797e-45d2-967e-f36d6959f28e admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:33.953768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa1d86-797e-45d2-967e-f36d6959f28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:33.954056 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa1d86-797e-45d2-967e-f36d6959f28e admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:33.954722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa1d86-797e-45d2-967e-f36d6959f28e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:33.956691 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa1d86-797e-45d2-967e-f36d6959f28e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:33.981996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1271/2534] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:33 2022] POST /identity/v3/users => generated 305 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:33.986399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b4f97-fc61-4332-992c-c45b4abd83fd admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:33.997712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b4f97-fc61-4332-992c-c45b4abd83fd admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:34.005211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a033e7-73b3-4a26-8a05-883b99bfc4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:34.005802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a033e7-73b3-4a26-8a05-883b99bfc4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:34.006119 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a033e7-73b3-4a26-8a05-883b99bfc4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:34.006395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a033e7-73b3-4a26-8a05-883b99bfc4d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:34.007060 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a033e7-73b3-4a26-8a05-883b99bfc4d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:34.008762 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a033e7-73b3-4a26-8a05-883b99bfc4d0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:34.025344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1264/2535] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:34.028953 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa1d86-797e-45d2-967e-f36d6959f28e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:34.047015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa1d86-797e-45d2-967e-f36d6959f28e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:34.054836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c79e6c5-1869-4611-ad76-31823f0a7635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:34.055476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79e6c5-1869-4611-ad76-31823f0a7635 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:34.055762 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79e6c5-1869-4611-ad76-31823f0a7635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:34.056029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c79e6c5-1869-4611-ad76-31823f0a7635 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:34.056693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79e6c5-1869-4611-ad76-31823f0a7635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:34.058340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c79e6c5-1869-4611-ad76-31823f0a7635 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:34.072012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1272/2536] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:34.076960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a033e7-73b3-4a26-8a05-883b99bfc4d0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:34.088776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a033e7-73b3-4a26-8a05-883b99bfc4d0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:34.097840 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20961942-2c71-499a-b516-5d827f42aefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:34.100399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20961942-2c71-499a-b516-5d827f42aefb admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:34.100399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20961942-2c71-499a-b516-5d827f42aefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:34.100399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20961942-2c71-499a-b516-5d827f42aefb admin admin] PATH_INFO: `/v3/projects/0ac22a83a8e04b6bb1afd31793110558/users/9d0aa28de6ae4152bf04e43dd71335aa/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:34.117313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20961942-2c71-499a-b516-5d827f42aefb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ac22a83a8e04b6bb1afd31793110558, user_id=9d0aa28de6ae4152bf04e43dd71335aa, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:34.121335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20961942-2c71-499a-b516-5d827f42aefb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:34.140842 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1265/2537] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:34 2022] PUT /identity/v3/projects/0ac22a83a8e04b6bb1afd31793110558/users/9d0aa28de6ae4152bf04e43dd71335aa/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:34.154364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa502ef9-a4d7-471e-876a-b19b98defd5f None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:34.154737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa502ef9-a4d7-471e-876a-b19b98defd5f None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:34.155049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa502ef9-a4d7-471e-876a-b19b98defd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:34.187909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-aa502ef9-a4d7-471e-876a-b19b98defd5f None None] MFA Rules not processed for user `9d0aa28de6ae4152bf04e43dd71335aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:34.189714 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-aa502ef9-a4d7-471e-876a-b19b98defd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:34.266424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:34.294192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1273/2538] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:34 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:34.299249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20961942-2c71-499a-b516-5d827f42aefb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:34.379956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20961942-2c71-499a-b516-5d827f42aefb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:34.386135 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1ee1a-eb44-471e-9446-4d4ad64ebf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:34.387130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1ee1a-eb44-471e-9446-4d4ad64ebf05 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:34.388207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1ee1a-eb44-471e-9446-4d4ad64ebf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:34.388766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1ee1a-eb44-471e-9446-4d4ad64ebf05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:34.389441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1ee1a-eb44-471e-9446-4d4ad64ebf05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:34.396674 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1ee1a-eb44-471e-9446-4d4ad64ebf05 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:34.427328 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:34.445225 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1266/2539] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:34 2022] POST /identity/v3/projects => generated 409 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:34.449987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa502ef9-a4d7-471e-876a-b19b98defd5f None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:34.464671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa502ef9-a4d7-471e-876a-b19b98defd5f None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:34.561472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb8a3b-66d9-4995-8839-9a2e8ef1bc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:34.562387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb8a3b-66d9-4995-8839-9a2e8ef1bc3d admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:34.562530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb8a3b-66d9-4995-8839-9a2e8ef1bc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:34.562797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb8a3b-66d9-4995-8839-9a2e8ef1bc3d admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:34.563534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb8a3b-66d9-4995-8839-9a2e8ef1bc3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:34.566845 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb8a3b-66d9-4995-8839-9a2e8ef1bc3d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:34.599369 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1274/2540] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:34 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:34.603392 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1ee1a-eb44-471e-9446-4d4ad64ebf05 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:34.619222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1ee1a-eb44-471e-9446-4d4ad64ebf05 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:34.636448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd21f1-9a1a-43f0-ad41-3a3ddd6dac29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:34.639305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd21f1-9a1a-43f0-ad41-3a3ddd6dac29 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:34.639586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd21f1-9a1a-43f0-ad41-3a3ddd6dac29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:34.639982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd21f1-9a1a-43f0-ad41-3a3ddd6dac29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:34.640515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd21f1-9a1a-43f0-ad41-3a3ddd6dac29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:34.647535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd21f1-9a1a-43f0-ad41-3a3ddd6dac29 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:34.669798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1267/2541] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:34.674501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb8a3b-66d9-4995-8839-9a2e8ef1bc3d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:34.690881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb8a3b-66d9-4995-8839-9a2e8ef1bc3d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:34.701165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35063dd0-1227-4e20-9bc2-bf37663bbc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:34.701861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35063dd0-1227-4e20-9bc2-bf37663bbc36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:34.702269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35063dd0-1227-4e20-9bc2-bf37663bbc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:34.702447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35063dd0-1227-4e20-9bc2-bf37663bbc36 admin admin] PATH_INFO: `/v3/projects/b187f65fb14440e3b9303f494d390b40/users/a04996ffc81a4164b1b1b9d9c67207f3/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:34.725415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35063dd0-1227-4e20-9bc2-bf37663bbc36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b187f65fb14440e3b9303f494d390b40, user_id=a04996ffc81a4164b1b1b9d9c67207f3, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:34.728295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35063dd0-1227-4e20-9bc2-bf37663bbc36 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:34.757244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1275/2542] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:34 2022] PUT /identity/v3/projects/b187f65fb14440e3b9303f494d390b40/users/a04996ffc81a4164b1b1b9d9c67207f3/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:34.763646 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd21f1-9a1a-43f0-ad41-3a3ddd6dac29 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:34.776786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd21f1-9a1a-43f0-ad41-3a3ddd6dac29 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:34.841937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070af9b-30b6-406a-8e19-989ab4121948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:34.842596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070af9b-30b6-406a-8e19-989ab4121948 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:34.842877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070af9b-30b6-406a-8e19-989ab4121948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:34.843146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070af9b-30b6-406a-8e19-989ab4121948 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:34.843790 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1070af9b-30b6-406a-8e19-989ab4121948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:34.846473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1070af9b-30b6-406a-8e19-989ab4121948 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:34.857742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1268/2543] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:34.863169 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35063dd0-1227-4e20-9bc2-bf37663bbc36 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:34.874910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35063dd0-1227-4e20-9bc2-bf37663bbc36 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:34.881790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dbefa-6bae-4624-97a0-33170fbc3571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:34.881790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dbefa-6bae-4624-97a0-33170fbc3571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:34.881790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dbefa-6bae-4624-97a0-33170fbc3571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:34.881790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dbefa-6bae-4624-97a0-33170fbc3571 admin admin] PATH_INFO: `/v3/projects/b187f65fb14440e3b9303f494d390b40/users/a04996ffc81a4164b1b1b9d9c67207f3/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:34.883644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880dbefa-6bae-4624-97a0-33170fbc3571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b187f65fb14440e3b9303f494d390b40, user_id=a04996ffc81a4164b1b1b9d9c67207f3, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:34.886208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880dbefa-6bae-4624-97a0-33170fbc3571 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:34.900968 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1276/2544] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:34 2022] PUT /identity/v3/projects/b187f65fb14440e3b9303f494d390b40/users/a04996ffc81a4164b1b1b9d9c67207f3/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:34.916192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2062c-ff35-42c1-a7f9-c8d8bd1118ac None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:34.916481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2062c-ff35-42c1-a7f9-c8d8bd1118ac None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:34.916769 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a2062c-ff35-42c1-a7f9-c8d8bd1118ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:34.952439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-36a2062c-ff35-42c1-a7f9-c8d8bd1118ac None None] MFA Rules not processed for user `a04996ffc81a4164b1b1b9d9c67207f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:34.953685 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-36a2062c-ff35-42c1-a7f9-c8d8bd1118ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:35.061542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1269/2545] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:34 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:35.070848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dbefa-6bae-4624-97a0-33170fbc3571 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:35.073783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-880dbefa-6bae-4624-97a0-33170fbc3571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:35.075886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2062c-ff35-42c1-a7f9-c8d8bd1118ac None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:35.077379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-36a2062c-ff35-42c1-a7f9-c8d8bd1118ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:35.154157 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a2062c-ff35-42c1-a7f9-c8d8bd1118ac None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:35.158793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59afc7-49f6-483d-ac5c-33501c36dba9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:35.159554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59afc7-49f6-483d-ac5c-33501c36dba9 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:35.160156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59afc7-49f6-483d-ac5c-33501c36dba9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:35.160485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e59afc7-49f6-483d-ac5c-33501c36dba9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:35.170257 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dbefa-6bae-4624-97a0-33170fbc3571 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:35.172820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59afc7-49f6-483d-ac5c-33501c36dba9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:35.174284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd3f79-fa40-44c9-9cac-c5b0a6e2dc77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:35.174521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e59afc7-49f6-483d-ac5c-33501c36dba9 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:35.175022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd3f79-fa40-44c9-9cac-c5b0a6e2dc77 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:35.175534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd3f79-fa40-44c9-9cac-c5b0a6e2dc77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:35.175721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd3f79-fa40-44c9-9cac-c5b0a6e2dc77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:35.177828 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a1cd3f79-fa40-44c9-9cac-c5b0a6e2dc77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:35.186259 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1270/2546] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:35 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:30:35.271617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:35.275833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd3f79-fa40-44c9-9cac-c5b0a6e2dc77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:35.278568 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd3f79-fa40-44c9-9cac-c5b0a6e2dc77 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:35.292039 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1277/2547] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:35.386624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59afc7-49f6-483d-ac5c-33501c36dba9 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:35.398006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e59afc7-49f6-483d-ac5c-33501c36dba9 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:35.404818 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e2dab-232b-4346-8a7c-e0033851359f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:35.405438 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e2dab-232b-4346-8a7c-e0033851359f service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:35.405723 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e2dab-232b-4346-8a7c-e0033851359f service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:35.405999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e2dab-232b-4346-8a7c-e0033851359f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:35.415844 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:35.486671 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e2dab-232b-4346-8a7c-e0033851359f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:35.489186 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e2dab-232b-4346-8a7c-e0033851359f service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:35.506189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1271/2548] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:35 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:39.011498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd3f79-fa40-44c9-9cac-c5b0a6e2dc77 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:39.013014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a1cd3f79-fa40-44c9-9cac-c5b0a6e2dc77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:39.036942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd3f79-fa40-44c9-9cac-c5b0a6e2dc77 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:39.043699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cf9e7-ffcd-4b58-b4d1-f65568e7eb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:39.044381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cf9e7-ffcd-4b58-b4d1-f65568e7eb13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:39.044694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cf9e7-ffcd-4b58-b4d1-f65568e7eb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:39.044931 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cf9e7-ffcd-4b58-b4d1-f65568e7eb13 admin admin] PATH_INFO: `/v3/users/b917d84a650f4355a676e315ea039f0e` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:39.046499 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cf9e7-ffcd-4b58-b4d1-f65568e7eb13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b917d84a650f4355a676e315ea039f0e)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:39.049506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cf9e7-ffcd-4b58-b4d1-f65568e7eb13 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:39.125637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-195cf9e7-ffcd-4b58-b4d1-f65568e7eb13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b917d84a650f4355a676e315ea039f0e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:39.134838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-195cf9e7-ffcd-4b58-b4d1-f65568e7eb13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b917d84a650f4355a676e315ea039f0e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:39.135827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-195cf9e7-ffcd-4b58-b4d1-f65568e7eb13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b917d84a650f4355a676e315ea039f0e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:39.146770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-195cf9e7-ffcd-4b58-b4d1-f65568e7eb13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b917d84a650f4355a676e315ea039f0e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:39.164135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-195cf9e7-ffcd-4b58-b4d1-f65568e7eb13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b917d84a650f4355a676e315ea039f0e'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:39.171464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1278/2549] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:39 2022] DELETE /identity/v3/users/b917d84a650f4355a676e315ea039f0e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:39.357346 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e2dab-232b-4346-8a7c-e0033851359f service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:39.369856 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e2dab-232b-4346-8a7c-e0033851359f service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:39.436689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:39.437380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:39.437664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:39.437939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec admin admin] PATH_INFO: `/v3/projects/ce7a3e77ddf14b48986752434dda6d33` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:39.439220 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce7a3e77ddf14b48986752434dda6d33)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:39.441191 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:39.493539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce7a3e77ddf14b48986752434dda6d33', 'request_id': 'req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:39.504290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce7a3e77ddf14b48986752434dda6d33', 'request_id': 'req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:39.513987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce7a3e77ddf14b48986752434dda6d33', 'request_id': 'req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:39.526803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec admin admin] The token cache is being invalidate because project ce7a3e77ddf14b48986752434dda6d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:39.527179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:39.528303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2463492d-d0ce-443b-b6b3-e2d0c5e545ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:39.529786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1272/2550] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:39 2022] DELETE /identity/v3/projects/ce7a3e77ddf14b48986752434dda6d33 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:39.534913 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fef16-acce-46b1-8b23-06e063dc2167 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:39.535204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fef16-acce-46b1-8b23-06e063dc2167 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:39.535480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2fef16-acce-46b1-8b23-06e063dc2167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:39.559153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2f2fef16-acce-46b1-8b23-06e063dc2167 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:39.560356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2f2fef16-acce-46b1-8b23-06e063dc2167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:39.619988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1279/2551] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:39.628279 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fc1b3-5307-4e0a-b685-044691fdba4f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:39.628573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fc1b3-5307-4e0a-b685-044691fdba4f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:39.628858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138fc1b3-5307-4e0a-b685-044691fdba4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:39.650482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-138fc1b3-5307-4e0a-b685-044691fdba4f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:39.651714 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-138fc1b3-5307-4e0a-b685-044691fdba4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:39.666092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1273/2552] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:39.668800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fef16-acce-46b1-8b23-06e063dc2167 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:39.684138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2fef16-acce-46b1-8b23-06e063dc2167 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:39.690326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd108af-4baa-4131-8010-af4a401e6225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:39.691464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd108af-4baa-4131-8010-af4a401e6225 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:39.691750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd108af-4baa-4131-8010-af4a401e6225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:39.692027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd108af-4baa-4131-8010-af4a401e6225 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:39.692693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd108af-4baa-4131-8010-af4a401e6225 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:39.693006 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd108af-4baa-4131-8010-af4a401e6225 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:39.694978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd108af-4baa-4131-8010-af4a401e6225 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:39.712639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1280/2553] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:30:39.718698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fc1b3-5307-4e0a-b685-044691fdba4f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:39.730016 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138fc1b3-5307-4e0a-b685-044691fdba4f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:39.734903 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5eb93e-d4c0-4391-8255-d90fecd283af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:39.735592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5eb93e-d4c0-4391-8255-d90fecd283af admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:39.735872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5eb93e-d4c0-4391-8255-d90fecd283af admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:39.736146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5eb93e-d4c0-4391-8255-d90fecd283af admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:39.736780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5eb93e-d4c0-4391-8255-d90fecd283af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:39.738735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5eb93e-d4c0-4391-8255-d90fecd283af admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:39.768232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1274/2554] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:39 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:39.772245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd108af-4baa-4131-8010-af4a401e6225 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:39.782256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd108af-4baa-4131-8010-af4a401e6225 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:39.831446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:39.842322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f98692-278d-487f-90c7-25745cc2ca37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:39.843130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f98692-278d-487f-90c7-25745cc2ca37 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:39.843823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f98692-278d-487f-90c7-25745cc2ca37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:39.844465 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f98692-278d-487f-90c7-25745cc2ca37 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:39.845187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f98692-278d-487f-90c7-25745cc2ca37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:39.847658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f98692-278d-487f-90c7-25745cc2ca37 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:39.876865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1281/2555] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:39 2022] POST /identity/v3/users => generated 303 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:39.877966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5eb93e-d4c0-4391-8255-d90fecd283af admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:39.889056 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5eb93e-d4c0-4391-8255-d90fecd283af admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:39.894012 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf3133-2999-4b51-ac22-35cc154bedb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:39.894634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf3133-2999-4b51-ac22-35cc154bedb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:39.894918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf3133-2999-4b51-ac22-35cc154bedb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:39.895195 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf3133-2999-4b51-ac22-35cc154bedb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:39.895837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf3133-2999-4b51-ac22-35cc154bedb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:39.897455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf3133-2999-4b51-ac22-35cc154bedb9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:39.900185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:39.909834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1275/2556] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:39.913893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f98692-278d-487f-90c7-25745cc2ca37 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:39.924504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f98692-278d-487f-90c7-25745cc2ca37 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:39.929097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6084ae46-9bae-4688-b6a4-0c6f776ef757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:39.930343 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084ae46-9bae-4688-b6a4-0c6f776ef757 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:39.930612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084ae46-9bae-4688-b6a4-0c6f776ef757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:39.930888 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6084ae46-9bae-4688-b6a4-0c6f776ef757 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:39.931535 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084ae46-9bae-4688-b6a4-0c6f776ef757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:39.933140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6084ae46-9bae-4688-b6a4-0c6f776ef757 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:39.945292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1282/2557] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:39.949496 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf3133-2999-4b51-ac22-35cc154bedb9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:39.959690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf3133-2999-4b51-ac22-35cc154bedb9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:39.964436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aaa63-5970-4ecd-b164-a66ef91de1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:39.965103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aaa63-5970-4ecd-b164-a66ef91de1f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:39.965391 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aaa63-5970-4ecd-b164-a66ef91de1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:39.965669 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aaa63-5970-4ecd-b164-a66ef91de1f0 admin admin] PATH_INFO: `/v3/projects/bdd7400366ed4d96b87eb0c74c35515c/users/c3a2504285bd4de1946181bef3f97877/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:39.981511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621aaa63-5970-4ecd-b164-a66ef91de1f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdd7400366ed4d96b87eb0c74c35515c, user_id=c3a2504285bd4de1946181bef3f97877, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:39.984332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621aaa63-5970-4ecd-b164-a66ef91de1f0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:40.000193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1276/2558] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:39 2022] PUT /identity/v3/projects/bdd7400366ed4d96b87eb0c74c35515c/users/c3a2504285bd4de1946181bef3f97877/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:40.010333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c661ab3-6396-46d1-b6b6-10567d5a09ee None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:40.010625 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c661ab3-6396-46d1-b6b6-10567d5a09ee None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:40.010903 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c661ab3-6396-46d1-b6b6-10567d5a09ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:40.040453 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-4c661ab3-6396-46d1-b6b6-10567d5a09ee None None] MFA Rules not processed for user `c3a2504285bd4de1946181bef3f97877`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:40.041673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-4c661ab3-6396-46d1-b6b6-10567d5a09ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:40.110090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1283/2559] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:40 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:40.111338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aaa63-5970-4ecd-b164-a66ef91de1f0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:40.181894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aaa63-5970-4ecd-b164-a66ef91de1f0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:40.187625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1026e-5b66-477a-b871-7db3d828df40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:40.188215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1026e-5b66-477a-b871-7db3d828df40 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:40.188496 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1026e-5b66-477a-b871-7db3d828df40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:40.188772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1026e-5b66-477a-b871-7db3d828df40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:40.189418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1026e-5b66-477a-b871-7db3d828df40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:40.191367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1026e-5b66-477a-b871-7db3d828df40 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:40.223072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1277/2560] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:40 2022] POST /identity/v3/projects => generated 403 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:40.227095 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c661ab3-6396-46d1-b6b6-10567d5a09ee None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:40.237658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c661ab3-6396-46d1-b6b6-10567d5a09ee None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:40.296434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73421e-39be-4237-a664-50a5de3269be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:40.297399 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73421e-39be-4237-a664-50a5de3269be admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:40.297914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73421e-39be-4237-a664-50a5de3269be admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:40.298401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c73421e-39be-4237-a664-50a5de3269be admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:40.299423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73421e-39be-4237-a664-50a5de3269be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:40.301737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c73421e-39be-4237-a664-50a5de3269be admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:40.326975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1284/2561] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:40 2022] POST /identity/v3/users => generated 308 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:40.331370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1026e-5b66-477a-b871-7db3d828df40 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:40.344281 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1026e-5b66-477a-b871-7db3d828df40 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:40.351710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237357f3-1c03-4119-8abd-46a27521fce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:40.353062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237357f3-1c03-4119-8abd-46a27521fce1 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:40.353643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237357f3-1c03-4119-8abd-46a27521fce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:40.354542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237357f3-1c03-4119-8abd-46a27521fce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:40.355794 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237357f3-1c03-4119-8abd-46a27521fce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:40.358219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237357f3-1c03-4119-8abd-46a27521fce1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:40.370832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1278/2562] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:40.375794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73421e-39be-4237-a664-50a5de3269be admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:40.386560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c73421e-39be-4237-a664-50a5de3269be admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:40.392273 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66099674-108d-4031-bfcd-ddf1b7722fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:40.393022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66099674-108d-4031-bfcd-ddf1b7722fac admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:40.393377 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66099674-108d-4031-bfcd-ddf1b7722fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:40.393725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66099674-108d-4031-bfcd-ddf1b7722fac admin admin] PATH_INFO: `/v3/projects/18cec8d874064e7286da1df81d46ac00/users/16c17dc8806547a2ada612df3442b0f6/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:40.411044 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66099674-108d-4031-bfcd-ddf1b7722fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cec8d874064e7286da1df81d46ac00, user_id=16c17dc8806547a2ada612df3442b0f6, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:40.413633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66099674-108d-4031-bfcd-ddf1b7722fac admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:40.430086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1285/2563] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:40 2022] PUT /identity/v3/projects/18cec8d874064e7286da1df81d46ac00/users/16c17dc8806547a2ada612df3442b0f6/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:40.435651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237357f3-1c03-4119-8abd-46a27521fce1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:40.460484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237357f3-1c03-4119-8abd-46a27521fce1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:40.536988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c59b3d-10f3-40a5-a604-30f886728ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:40.537633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c59b3d-10f3-40a5-a604-30f886728ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:40.537921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c59b3d-10f3-40a5-a604-30f886728ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:40.538232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c59b3d-10f3-40a5-a604-30f886728ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:40.538869 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c59b3d-10f3-40a5-a604-30f886728ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:40.540446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c59b3d-10f3-40a5-a604-30f886728ef8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:40.550908 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1279/2564] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:40.559599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66099674-108d-4031-bfcd-ddf1b7722fac admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:40.571995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66099674-108d-4031-bfcd-ddf1b7722fac admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:40.577376 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e7a7b-0743-48f2-a369-d3500fbff080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:40.578074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e7a7b-0743-48f2-a369-d3500fbff080 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:40.578354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e7a7b-0743-48f2-a369-d3500fbff080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:40.578628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e7a7b-0743-48f2-a369-d3500fbff080 admin admin] PATH_INFO: `/v3/projects/18cec8d874064e7286da1df81d46ac00/users/16c17dc8806547a2ada612df3442b0f6/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:40.580750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e7a7b-0743-48f2-a369-d3500fbff080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cec8d874064e7286da1df81d46ac00, user_id=16c17dc8806547a2ada612df3442b0f6, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:40.584020 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e7a7b-0743-48f2-a369-d3500fbff080 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:40.595448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1286/2565] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:40 2022] PUT /identity/v3/projects/18cec8d874064e7286da1df81d46ac00/users/16c17dc8806547a2ada612df3442b0f6/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:40.607252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c89eaa-7512-4f94-a6ba-11d2791cc50e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:40.607541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c89eaa-7512-4f94-a6ba-11d2791cc50e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:40.607816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c89eaa-7512-4f94-a6ba-11d2791cc50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:40.638451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f3c89eaa-7512-4f94-a6ba-11d2791cc50e None None] MFA Rules not processed for user `16c17dc8806547a2ada612df3442b0f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:40.639870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f3c89eaa-7512-4f94-a6ba-11d2791cc50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:40.678997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:40.751623 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1280/2566] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:40 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:40.779297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e7a7b-0743-48f2-a369-d3500fbff080 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:40.781049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5d5e7a7b-0743-48f2-a369-d3500fbff080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:40.797458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:40.849797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e7a7b-0743-48f2-a369-d3500fbff080 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:40.853740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2795f87-ea42-4f8c-add6-9ba61bdd22f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:40.854371 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2795f87-ea42-4f8c-add6-9ba61bdd22f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:40.854642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2795f87-ea42-4f8c-add6-9ba61bdd22f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:40.854924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2795f87-ea42-4f8c-add6-9ba61bdd22f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:40.868762 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2795f87-ea42-4f8c-add6-9ba61bdd22f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:40.870575 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2795f87-ea42-4f8c-add6-9ba61bdd22f6 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:40.882517 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1287/2567] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:40 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:41.115815 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c89eaa-7512-4f94-a6ba-11d2791cc50e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:41.207925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c89eaa-7512-4f94-a6ba-11d2791cc50e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:41.212869 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:41.213519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:41.213803 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:41.214962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] PATH_INFO: `/v3/users/c3a2504285bd4de1946181bef3f97877` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:41.215470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a2504285bd4de1946181bef3f97877)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:41.217401 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:41.280299 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a2504285bd4de1946181bef3f97877'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:41.288288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a2504285bd4de1946181bef3f97877'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:41.305362 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a2504285bd4de1946181bef3f97877'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:41.305362 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a2504285bd4de1946181bef3f97877'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:41.312434 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a2504285bd4de1946181bef3f97877'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:41.319523 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1281/2568] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:41 2022] DELETE /identity/v3/users/c3a2504285bd4de1946181bef3f97877 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:41.323781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb12df-da56-4deb-a267-7cc5a2068235 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:41.324893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb12df-da56-4deb-a267-7cc5a2068235 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:41.325003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb12df-da56-4deb-a267-7cc5a2068235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:41.347596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-dbeb12df-da56-4deb-a267-7cc5a2068235 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:41.348865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-dbeb12df-da56-4deb-a267-7cc5a2068235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:41.426161 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1288/2569] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:41.443128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:41.444748 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:41.513969 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:41.526641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a687749-ddf9-4cc2-b33d-faf34acdb6ee admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:41.526641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33c06c-de89-4db3-bf54-8c4fe82d24e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:41.526641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33c06c-de89-4db3-bf54-8c4fe82d24e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:41.526641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33c06c-de89-4db3-bf54-8c4fe82d24e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:41.527988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33c06c-de89-4db3-bf54-8c4fe82d24e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:41.535155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33c06c-de89-4db3-bf54-8c4fe82d24e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:41.536606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33c06c-de89-4db3-bf54-8c4fe82d24e8 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:41.548212 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1282/2570] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:41.854687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb12df-da56-4deb-a267-7cc5a2068235 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:41.856867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-dbeb12df-da56-4deb-a267-7cc5a2068235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:41.869194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb12df-da56-4deb-a267-7cc5a2068235 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:41.873720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:41.874387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:41.874664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:41.874936 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] PATH_INFO: `/v3/projects/bdd7400366ed4d96b87eb0c74c35515c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:41.876172 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdd7400366ed4d96b87eb0c74c35515c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:41.878072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:41.927871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdd7400366ed4d96b87eb0c74c35515c', 'request_id': 'req-900b6a8f-5598-4860-894c-6de18a6f9c46'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:41.934943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdd7400366ed4d96b87eb0c74c35515c', 'request_id': 'req-900b6a8f-5598-4860-894c-6de18a6f9c46'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:41.941205 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdd7400366ed4d96b87eb0c74c35515c', 'request_id': 'req-900b6a8f-5598-4860-894c-6de18a6f9c46'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:41.952019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] The token cache is being invalidate because project bdd7400366ed4d96b87eb0c74c35515c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:41.952311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:41.952970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:41.954485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1289/2571] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:41 2022] DELETE /identity/v3/projects/bdd7400366ed4d96b87eb0c74c35515c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:41.958463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33c06c-de89-4db3-bf54-8c4fe82d24e8 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:41.968440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33c06c-de89-4db3-bf54-8c4fe82d24e8 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:42.025370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2bbcbd-a91e-470c-9bd0-9b3b872ee089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:42.025997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bbcbd-a91e-470c-9bd0-9b3b872ee089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:42.026372 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bbcbd-a91e-470c-9bd0-9b3b872ee089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:42.026866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2bbcbd-a91e-470c-9bd0-9b3b872ee089 admin admin] PATH_INFO: `/v3/users/16c17dc8806547a2ada612df3442b0f6` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:42.028581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bbcbd-a91e-470c-9bd0-9b3b872ee089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16c17dc8806547a2ada612df3442b0f6)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:42.030516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2bbcbd-a91e-470c-9bd0-9b3b872ee089 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:42.082109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd2bbcbd-a91e-470c-9bd0-9b3b872ee089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16c17dc8806547a2ada612df3442b0f6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:42.092102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd2bbcbd-a91e-470c-9bd0-9b3b872ee089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16c17dc8806547a2ada612df3442b0f6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:42.109843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd2bbcbd-a91e-470c-9bd0-9b3b872ee089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16c17dc8806547a2ada612df3442b0f6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:42.110736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd2bbcbd-a91e-470c-9bd0-9b3b872ee089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16c17dc8806547a2ada612df3442b0f6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:42.119288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fd2bbcbd-a91e-470c-9bd0-9b3b872ee089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16c17dc8806547a2ada612df3442b0f6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:42.126104 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1283/2572] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:41 2022] DELETE /identity/v3/users/16c17dc8806547a2ada612df3442b0f6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:42.417623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:42.419157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:42.434038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b6a8f-5598-4860-894c-6de18a6f9c46 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:42.480225 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:42.502140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69771a7f-9eb3-49fd-bd9c-98ddad27c525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:42.502822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69771a7f-9eb3-49fd-bd9c-98ddad27c525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:42.503105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69771a7f-9eb3-49fd-bd9c-98ddad27c525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:42.503381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69771a7f-9eb3-49fd-bd9c-98ddad27c525 admin admin] PATH_INFO: `/v3/projects/18cec8d874064e7286da1df81d46ac00` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:42.504679 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69771a7f-9eb3-49fd-bd9c-98ddad27c525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18cec8d874064e7286da1df81d46ac00)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:42.506681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69771a7f-9eb3-49fd-bd9c-98ddad27c525 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:42.557843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-69771a7f-9eb3-49fd-bd9c-98ddad27c525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18cec8d874064e7286da1df81d46ac00', 'request_id': 'req-69771a7f-9eb3-49fd-bd9c-98ddad27c525'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:42.565719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-69771a7f-9eb3-49fd-bd9c-98ddad27c525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18cec8d874064e7286da1df81d46ac00', 'request_id': 'req-69771a7f-9eb3-49fd-bd9c-98ddad27c525'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:42.573326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-69771a7f-9eb3-49fd-bd9c-98ddad27c525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18cec8d874064e7286da1df81d46ac00', 'request_id': 'req-69771a7f-9eb3-49fd-bd9c-98ddad27c525'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:42.590577 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-69771a7f-9eb3-49fd-bd9c-98ddad27c525 admin admin] The token cache is being invalidate because project 18cec8d874064e7286da1df81d46ac00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:42.591092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-69771a7f-9eb3-49fd-bd9c-98ddad27c525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:42.592136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-69771a7f-9eb3-49fd-bd9c-98ddad27c525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:42.594513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1290/2573] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:42 2022] DELETE /identity/v3/projects/18cec8d874064e7286da1df81d46ac00 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:42.599511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090930c7-9e54-4f7e-8882-f6012e70c9b6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:42.599999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090930c7-9e54-4f7e-8882-f6012e70c9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:42.600098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090930c7-9e54-4f7e-8882-f6012e70c9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:42.622763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-090930c7-9e54-4f7e-8882-f6012e70c9b6 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:42.624049 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-090930c7-9e54-4f7e-8882-f6012e70c9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:42.701017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1284/2574] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:42.709496 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821f140-cf1e-4118-9ded-6266b91cbca2 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:42.709962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821f140-cf1e-4118-9ded-6266b91cbca2 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:42.710494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b821f140-cf1e-4118-9ded-6266b91cbca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:42.735445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b821f140-cf1e-4118-9ded-6266b91cbca2 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:42.736678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b821f140-cf1e-4118-9ded-6266b91cbca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:42.748324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1291/2575] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:42.752770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090930c7-9e54-4f7e-8882-f6012e70c9b6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:42.764608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090930c7-9e54-4f7e-8882-f6012e70c9b6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:42.770579 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3afab-f65b-4863-bd1a-8f6fae2fc827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:42.771210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3afab-f65b-4863-bd1a-8f6fae2fc827 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:42.771491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3afab-f65b-4863-bd1a-8f6fae2fc827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:42.771882 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3afab-f65b-4863-bd1a-8f6fae2fc827 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:42.772572 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3afab-f65b-4863-bd1a-8f6fae2fc827 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:42.772876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3afab-f65b-4863-bd1a-8f6fae2fc827 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:42.774573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3afab-f65b-4863-bd1a-8f6fae2fc827 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:42.784576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:42.795918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821f140-cf1e-4118-9ded-6266b91cbca2 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:42.798176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1285/2576] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:30:42.812183 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b821f140-cf1e-4118-9ded-6266b91cbca2 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:42.821528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dfed3-74ea-4a1a-bdd2-c9cf78cb507f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:42.822782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dfed3-74ea-4a1a-bdd2-c9cf78cb507f admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:42.823275 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dfed3-74ea-4a1a-bdd2-c9cf78cb507f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:42.823758 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dfed3-74ea-4a1a-bdd2-c9cf78cb507f admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:42.825126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6dfed3-74ea-4a1a-bdd2-c9cf78cb507f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:42.828829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6dfed3-74ea-4a1a-bdd2-c9cf78cb507f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:42.866812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1292/2577] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:42 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:42.870984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3afab-f65b-4863-bd1a-8f6fae2fc827 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:42.881385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3afab-f65b-4863-bd1a-8f6fae2fc827 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:42.946899 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d9521-f565-49fb-8125-22fd651b3e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:42.947528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d9521-f565-49fb-8125-22fd651b3e73 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:42.947842 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d9521-f565-49fb-8125-22fd651b3e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:42.948143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d9521-f565-49fb-8125-22fd651b3e73 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:42.948780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d9521-f565-49fb-8125-22fd651b3e73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:42.951300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d9521-f565-49fb-8125-22fd651b3e73 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:42.974317 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1286/2578] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:42 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:42.978246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dfed3-74ea-4a1a-bdd2-c9cf78cb507f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:42.990959 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dfed3-74ea-4a1a-bdd2-c9cf78cb507f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:42.996912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63555283-48e3-4826-b717-0f6db1063fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:42.997727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63555283-48e3-4826-b717-0f6db1063fce admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:42.998029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63555283-48e3-4826-b717-0f6db1063fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:42.998358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63555283-48e3-4826-b717-0f6db1063fce admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:42.999112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63555283-48e3-4826-b717-0f6db1063fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:43.000800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63555283-48e3-4826-b717-0f6db1063fce admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:43.012008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1293/2579] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:43.016125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d9521-f565-49fb-8125-22fd651b3e73 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:43.026370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d9521-f565-49fb-8125-22fd651b3e73 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:43.031679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb043d9f-2d61-4e1a-b37f-5e4c20040f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:43.032287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb043d9f-2d61-4e1a-b37f-5e4c20040f6f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:43.032566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb043d9f-2d61-4e1a-b37f-5e4c20040f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:43.032838 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb043d9f-2d61-4e1a-b37f-5e4c20040f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:43.033483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb043d9f-2d61-4e1a-b37f-5e4c20040f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:43.035149 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb043d9f-2d61-4e1a-b37f-5e4c20040f6f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:43.049664 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1287/2580] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:43.053982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63555283-48e3-4826-b717-0f6db1063fce admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:43.064372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63555283-48e3-4826-b717-0f6db1063fce admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:43.068855 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9312b-8c3a-4d08-bba6-c750ed66d974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:43.069540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9312b-8c3a-4d08-bba6-c750ed66d974 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:43.069819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9312b-8c3a-4d08-bba6-c750ed66d974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:43.070114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9312b-8c3a-4d08-bba6-c750ed66d974 admin admin] PATH_INFO: `/v3/projects/760e8a3fbe3b4b6da8312017eb7af2b0/users/38a8024cdd55440e83e3a521621ed6dd/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:43.086143 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9312b-8c3a-4d08-bba6-c750ed66d974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=760e8a3fbe3b4b6da8312017eb7af2b0, user_id=38a8024cdd55440e83e3a521621ed6dd, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:43.090129 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9312b-8c3a-4d08-bba6-c750ed66d974 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:43.103483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1294/2581] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:43 2022] PUT /identity/v3/projects/760e8a3fbe3b4b6da8312017eb7af2b0/users/38a8024cdd55440e83e3a521621ed6dd/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:43.113918 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db46c6c-feca-49a0-8838-976bad316690 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:43.114245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db46c6c-feca-49a0-8838-976bad316690 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:43.114520 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db46c6c-feca-49a0-8838-976bad316690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:43.148121 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4db46c6c-feca-49a0-8838-976bad316690 None None] MFA Rules not processed for user `38a8024cdd55440e83e3a521621ed6dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:43.149432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4db46c6c-feca-49a0-8838-976bad316690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:43.241697 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1288/2582] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:43.247839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9312b-8c3a-4d08-bba6-c750ed66d974 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:43.322033 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9312b-8c3a-4d08-bba6-c750ed66d974 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:43.328075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd03c36-8a0b-4637-959c-f41964563952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:43.328809 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd03c36-8a0b-4637-959c-f41964563952 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:43.329171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd03c36-8a0b-4637-959c-f41964563952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:43.329521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd03c36-8a0b-4637-959c-f41964563952 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:43.330276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd03c36-8a0b-4637-959c-f41964563952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:43.332282 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd03c36-8a0b-4637-959c-f41964563952 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:43.345978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:43.370397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1295/2583] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:43 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:43.371931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db46c6c-feca-49a0-8838-976bad316690 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:43.384197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db46c6c-feca-49a0-8838-976bad316690 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:43.405331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd03c36-8a0b-4637-959c-f41964563952 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:43.407230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-bfd03c36-8a0b-4637-959c-f41964563952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:43.488399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a135a76-0b5c-4a4c-b595-ac54cc62ef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:43.489129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a135a76-0b5c-4a4c-b595-ac54cc62ef65 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:43.489886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a135a76-0b5c-4a4c-b595-ac54cc62ef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:43.490020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a135a76-0b5c-4a4c-b595-ac54cc62ef65 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:43.490717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a135a76-0b5c-4a4c-b595-ac54cc62ef65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:43.493094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a135a76-0b5c-4a4c-b595-ac54cc62ef65 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:43.505941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd03c36-8a0b-4637-959c-f41964563952 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:43.515107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:43.517957 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e406367-cd39-46d2-b0b5-3b2f0d3ca386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:43.518876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e406367-cd39-46d2-b0b5-3b2f0d3ca386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:43.519232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e406367-cd39-46d2-b0b5-3b2f0d3ca386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:43.519576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e406367-cd39-46d2-b0b5-3b2f0d3ca386 admin admin] PATH_INFO: `/v3/users/9d0aa28de6ae4152bf04e43dd71335aa` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:43.521109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e406367-cd39-46d2-b0b5-3b2f0d3ca386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d0aa28de6ae4152bf04e43dd71335aa)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:43.523483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e406367-cd39-46d2-b0b5-3b2f0d3ca386 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:43.529938 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1289/2584] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:43 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:43.534761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a135a76-0b5c-4a4c-b595-ac54cc62ef65 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:43.549876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a135a76-0b5c-4a4c-b595-ac54cc62ef65 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:43.556921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa5696-3635-49ef-b499-ad1eb5092a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:43.557706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa5696-3635-49ef-b499-ad1eb5092a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:43.558122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa5696-3635-49ef-b499-ad1eb5092a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:43.559578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa5696-3635-49ef-b499-ad1eb5092a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:43.559578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa5696-3635-49ef-b499-ad1eb5092a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:43.561833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa5696-3635-49ef-b499-ad1eb5092a10 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:43.578387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1290/2585] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:43.582446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa5696-3635-49ef-b499-ad1eb5092a10 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:43.594104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6e406367-cd39-46d2-b0b5-3b2f0d3ca386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d0aa28de6ae4152bf04e43dd71335aa'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:43.595313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa5696-3635-49ef-b499-ad1eb5092a10 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:43.602533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710a9b5-1c17-450d-bce0-8902f97be98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:43.603249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710a9b5-1c17-450d-bce0-8902f97be98a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:43.603249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710a9b5-1c17-450d-bce0-8902f97be98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:43.603518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4710a9b5-1c17-450d-bce0-8902f97be98a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:43.604150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710a9b5-1c17-450d-bce0-8902f97be98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:43.605993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6e406367-cd39-46d2-b0b5-3b2f0d3ca386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d0aa28de6ae4152bf04e43dd71335aa'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:43.606395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4710a9b5-1c17-450d-bce0-8902f97be98a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:43.607164 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6e406367-cd39-46d2-b0b5-3b2f0d3ca386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d0aa28de6ae4152bf04e43dd71335aa'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:43.619331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6e406367-cd39-46d2-b0b5-3b2f0d3ca386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d0aa28de6ae4152bf04e43dd71335aa'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:43.622568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1291/2586] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:43.624128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710a9b5-1c17-450d-bce0-8902f97be98a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:43.626421 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4710a9b5-1c17-450d-bce0-8902f97be98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:43.638898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6e406367-cd39-46d2-b0b5-3b2f0d3ca386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d0aa28de6ae4152bf04e43dd71335aa'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:43.645688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1296/2587] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:43 2022] DELETE /identity/v3/users/9d0aa28de6ae4152bf04e43dd71335aa => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:43.648860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4710a9b5-1c17-450d-bce0-8902f97be98a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:43.650379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8aec48-aeb7-4c18-a3cb-61208b4010fb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:43.650597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8aec48-aeb7-4c18-a3cb-61208b4010fb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:43.650796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8aec48-aeb7-4c18-a3cb-61208b4010fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:43.673510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ef8aec48-aeb7-4c18-a3cb-61208b4010fb None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:43.675045 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ef8aec48-aeb7-4c18-a3cb-61208b4010fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:43.771756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1297/2588] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:43.782686 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8aec48-aeb7-4c18-a3cb-61208b4010fb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:43.784415 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ef8aec48-aeb7-4c18-a3cb-61208b4010fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:43.803747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:43.847567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b4457-3591-461d-b3a3-c4b2ce1f9e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:43.848280 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b4457-3591-461d-b3a3-c4b2ce1f9e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:43.849021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b4457-3591-461d-b3a3-c4b2ce1f9e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:43.849021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b4457-3591-461d-b3a3-c4b2ce1f9e5e admin admin] PATH_INFO: `/v3/projects/ccfd93f5d8274c069ff1250f669933a1/users/ccea7d846ad342d6b86f10a3c372ac9f/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:43.864983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b4457-3591-461d-b3a3-c4b2ce1f9e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccfd93f5d8274c069ff1250f669933a1, user_id=ccea7d846ad342d6b86f10a3c372ac9f, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:43.867837 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b4457-3591-461d-b3a3-c4b2ce1f9e5e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:43.874514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8aec48-aeb7-4c18-a3cb-61208b4010fb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:43.878464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08061a-16db-4962-9e38-a8c42ef3850c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:43.879164 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08061a-16db-4962-9e38-a8c42ef3850c service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:43.879984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08061a-16db-4962-9e38-a8c42ef3850c service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:43.879984 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f08061a-16db-4962-9e38-a8c42ef3850c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:43.891170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08061a-16db-4962-9e38-a8c42ef3850c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:43.892672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f08061a-16db-4962-9e38-a8c42ef3850c service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:43.894128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1292/2589] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:43 2022] PUT /identity/v3/projects/ccfd93f5d8274c069ff1250f669933a1/users/ccea7d846ad342d6b86f10a3c372ac9f/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:43.907210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfc7eb-84e2-45fa-98ca-acdfe4aafc20 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:43.907515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfc7eb-84e2-45fa-98ca-acdfe4aafc20 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:43.907770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfc7eb-84e2-45fa-98ca-acdfe4aafc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:43.908671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1298/2590] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:43.939273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fdcfc7eb-84e2-45fa-98ca-acdfe4aafc20 None None] MFA Rules not processed for user `ccea7d846ad342d6b86f10a3c372ac9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:43.940546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fdcfc7eb-84e2-45fa-98ca-acdfe4aafc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:44.029126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1293/2591] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:44.033753 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08061a-16db-4962-9e38-a8c42ef3850c service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:44.044801 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f08061a-16db-4962-9e38-a8c42ef3850c service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:44.090177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfc7eb-84e2-45fa-98ca-acdfe4aafc20 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:44.091956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fdcfc7eb-84e2-45fa-98ca-acdfe4aafc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:44.178113 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e82d45-ca24-4c48-b10e-8a428a980691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:44.179817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e82d45-ca24-4c48-b10e-8a428a980691 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:44.179817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e82d45-ca24-4c48-b10e-8a428a980691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:44.180213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e82d45-ca24-4c48-b10e-8a428a980691 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:44.180861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e82d45-ca24-4c48-b10e-8a428a980691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:44.184619 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e82d45-ca24-4c48-b10e-8a428a980691 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:44.192860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:44.215437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfc7eb-84e2-45fa-98ca-acdfe4aafc20 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:44.221537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:44.222333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:44.222692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:44.223069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] PATH_INFO: `/v3/users/37a46ac5cb774189a4cca5f6880c646c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:44.224406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37a46ac5cb774189a4cca5f6880c646c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:44.226643 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:44.234307 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1299/2592] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:44 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:44.237611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e82d45-ca24-4c48-b10e-8a428a980691 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:44.307753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37a46ac5cb774189a4cca5f6880c646c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.321314 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37a46ac5cb774189a4cca5f6880c646c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.337715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37a46ac5cb774189a4cca5f6880c646c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.338455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37a46ac5cb774189a4cca5f6880c646c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.344745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e82d45-ca24-4c48-b10e-8a428a980691 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:44.349349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37a46ac5cb774189a4cca5f6880c646c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.350622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6bb98-118b-483b-bcb5-ea79e162cd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:44.351387 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6bb98-118b-483b-bcb5-ea79e162cd4a admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:44.351773 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6bb98-118b-483b-bcb5-ea79e162cd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:44.352640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e6bb98-118b-483b-bcb5-ea79e162cd4a admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:44.353419 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6bb98-118b-483b-bcb5-ea79e162cd4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:44.356209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e6bb98-118b-483b-bcb5-ea79e162cd4a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:44.357456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1294/2593] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:44 2022] DELETE /identity/v3/users/37a46ac5cb774189a4cca5f6880c646c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:44.387677 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1300/2594] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:44 2022] POST /identity/v3/users => generated 295 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:44.388430 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:44.391508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:44.463328 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6bb98-118b-483b-bcb5-ea79e162cd4a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:44.466503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f5e6bb98-118b-483b-bcb5-ea79e162cd4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:44.483092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e6bb98-118b-483b-bcb5-ea79e162cd4a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:44.525505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:44.530297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c9050-5a8d-4753-90b2-d4a49c0e0a35 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:44.536622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f79fe-7579-41dd-82fd-fdda77815610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:44.537241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f79fe-7579-41dd-82fd-fdda77815610 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:44.537510 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f79fe-7579-41dd-82fd-fdda77815610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:44.537776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f79fe-7579-41dd-82fd-fdda77815610 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:44.538758 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f79fe-7579-41dd-82fd-fdda77815610 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:44.540424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f79fe-7579-41dd-82fd-fdda77815610 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:44.551928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1295/2595] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:44.556905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f79fe-7579-41dd-82fd-fdda77815610 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:44.567657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f79fe-7579-41dd-82fd-fdda77815610 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:44.574237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:44.574994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:44.575523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:44.576121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] PATH_INFO: `/v3/projects/0ac22a83a8e04b6bb1afd31793110558` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:44.577095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30ab9c-734b-49cd-b87f-1c9fba2ae897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:44.577874 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30ab9c-734b-49cd-b87f-1c9fba2ae897 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:44.578285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30ab9c-734b-49cd-b87f-1c9fba2ae897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:44.578414 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ac22a83a8e04b6bb1afd31793110558)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:44.578554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30ab9c-734b-49cd-b87f-1c9fba2ae897 admin admin] PATH_INFO: `/v3/projects/2cd56e174bb941e0b5dc91d099d3d812/users/8b2bf77df28b494586da64d63fe18c74/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:44.582706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:44.606709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30ab9c-734b-49cd-b87f-1c9fba2ae897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cd56e174bb941e0b5dc91d099d3d812, user_id=8b2bf77df28b494586da64d63fe18c74, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:44.609684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e30ab9c-734b-49cd-b87f-1c9fba2ae897 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:44.626312 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1296/2596] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:44 2022] PUT /identity/v3/projects/2cd56e174bb941e0b5dc91d099d3d812/users/8b2bf77df28b494586da64d63fe18c74/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:44.630111 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30ab9c-734b-49cd-b87f-1c9fba2ae897 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:44.632163 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0e30ab9c-734b-49cd-b87f-1c9fba2ae897 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:44.652811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ac22a83a8e04b6bb1afd31793110558', 'request_id': 'req-a7183be8-f486-4feb-85ea-e9393c8ef1c9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.662723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ac22a83a8e04b6bb1afd31793110558', 'request_id': 'req-a7183be8-f486-4feb-85ea-e9393c8ef1c9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.672820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ac22a83a8e04b6bb1afd31793110558', 'request_id': 'req-a7183be8-f486-4feb-85ea-e9393c8ef1c9'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.678498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:44.686745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] The token cache is being invalidate because project 0ac22a83a8e04b6bb1afd31793110558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:44.687278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.688219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.691381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1301/2597] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:44 2022] DELETE /identity/v3/projects/0ac22a83a8e04b6bb1afd31793110558 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:44.692867 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:44.695640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:44.708399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30ab9c-734b-49cd-b87f-1c9fba2ae897 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:44.771542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7183be8-f486-4feb-85ea-e9393c8ef1c9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:44.776292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487a55b-3071-42e5-9384-1223b2fea4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:44.778457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487a55b-3071-42e5-9384-1223b2fea4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:44.778457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487a55b-3071-42e5-9384-1223b2fea4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:44.778457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487a55b-3071-42e5-9384-1223b2fea4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:44.778800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487a55b-3071-42e5-9384-1223b2fea4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:44.780489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487a55b-3071-42e5-9384-1223b2fea4d8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:44.790015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:44.790855 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1302/2598] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:44.791082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:44.791391 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:44.791530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487a55b-3071-42e5-9384-1223b2fea4d8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:44.791625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] PATH_INFO: `/v3/projects/661e8005aba1407d8aa0e760fbbaeee0` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:44.793472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] RBAC: Authorizing `identity:delete_project(project_id=661e8005aba1407d8aa0e760fbbaeee0)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:44.795689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3487a55b-3071-42e5-9384-1223b2fea4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:44.795854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:44.808250 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487a55b-3071-42e5-9384-1223b2fea4d8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:44.813170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:44.813902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:44.814221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:44.814522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] PATH_INFO: `/v3/users/a04996ffc81a4164b1b1b9d9c67207f3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:44.815739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a04996ffc81a4164b1b1b9d9c67207f3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:44.817753 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:44.856606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '661e8005aba1407d8aa0e760fbbaeee0', 'request_id': 'req-55ebc1f4-032b-4d3b-860b-02336776787e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.866392 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '661e8005aba1407d8aa0e760fbbaeee0', 'request_id': 'req-55ebc1f4-032b-4d3b-860b-02336776787e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.868980 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a04996ffc81a4164b1b1b9d9c67207f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.871717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:44.873248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '661e8005aba1407d8aa0e760fbbaeee0', 'request_id': 'req-55ebc1f4-032b-4d3b-860b-02336776787e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.878373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a04996ffc81a4164b1b1b9d9c67207f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.878859 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a04996ffc81a4164b1b1b9d9c67207f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.885699 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] The token cache is being invalidate because project 661e8005aba1407d8aa0e760fbbaeee0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:44.885985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.886942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.888180 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a04996ffc81a4164b1b1b9d9c67207f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.889221 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1297/2599] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:44 2022] DELETE /identity/v3/projects/661e8005aba1407d8aa0e760fbbaeee0 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:44.889988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:44.891642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:44.903785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a04996ffc81a4164b1b1b9d9c67207f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:44.911512 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1303/2600] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:44 2022] DELETE /identity/v3/users/a04996ffc81a4164b1b1b9d9c67207f3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:44.912632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:44.914025 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:44.994619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ebc1f4-032b-4d3b-860b-02336776787e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:45.016551 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:45.028222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a083386-2909-4f69-acde-bb817949a25f admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:45.036327 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:45.037599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.038849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.038849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] PATH_INFO: `/v3/users/437ea6e388fd47d78f9d9935ae4c5cfe` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.040216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] RBAC: Authorizing `identity:delete_user(user_id=437ea6e388fd47d78f9d9935ae4c5cfe)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:45.043236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:45.075619 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6e8b2d-d475-49da-b2f5-c7c694133135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:45.076338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e8b2d-d475-49da-b2f5-c7c694133135 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.076614 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e8b2d-d475-49da-b2f5-c7c694133135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.076934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6e8b2d-d475-49da-b2f5-c7c694133135 admin admin] PATH_INFO: `/v3/projects/2cd56e174bb941e0b5dc91d099d3d812/users/8b2bf77df28b494586da64d63fe18c74/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.079395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e8b2d-d475-49da-b2f5-c7c694133135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cd56e174bb941e0b5dc91d099d3d812, user_id=8b2bf77df28b494586da64d63fe18c74, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:45.082211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6e8b2d-d475-49da-b2f5-c7c694133135 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:45.110081 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1298/2601] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:44 2022] PUT /identity/v3/projects/2cd56e174bb941e0b5dc91d099d3d812/users/8b2bf77df28b494586da64d63fe18c74/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:45.116530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5035d-75eb-4ed0-b807-a5d714b3259d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.116795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5035d-75eb-4ed0-b807-a5d714b3259d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.117168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5035d-75eb-4ed0-b807-a5d714b3259d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.119443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '437ea6e388fd47d78f9d9935ae4c5cfe'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.139849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '437ea6e388fd47d78f9d9935ae4c5cfe'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.141181 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '437ea6e388fd47d78f9d9935ae4c5cfe'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.154453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e2f5035d-75eb-4ed0-b807-a5d714b3259d None None] MFA Rules not processed for user `8b2bf77df28b494586da64d63fe18c74`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:45.154884 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '437ea6e388fd47d78f9d9935ae4c5cfe'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.155881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e2f5035d-75eb-4ed0-b807-a5d714b3259d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:45.174616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '437ea6e388fd47d78f9d9935ae4c5cfe'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.181570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1304/2602] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:44 2022] DELETE /identity/v3/users/437ea6e388fd47d78f9d9935ae4c5cfe => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:45.260338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1299/2603] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:45.272636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:45.274203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:45.284871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5035d-75eb-4ed0-b807-a5d714b3259d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:45.288088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e2f5035d-75eb-4ed0-b807-a5d714b3259d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:45.353446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:45.363156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfcbba8-4f9a-45b4-a50e-d0f994c471be admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:45.371740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6cc98-03f7-441c-9a76-7c0c78f09a74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:45.372294 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6cc98-03f7-441c-9a76-7c0c78f09a74 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.372576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6cc98-03f7-441c-9a76-7c0c78f09a74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.372851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6cc98-03f7-441c-9a76-7c0c78f09a74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.381827 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5035d-75eb-4ed0-b807-a5d714b3259d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:45.442812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:45.452951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6cc98-03f7-441c-9a76-7c0c78f09a74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:45.454511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6cc98-03f7-441c-9a76-7c0c78f09a74 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:45.456972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:45.457751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.458032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.458341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] PATH_INFO: `/v3/projects/b187f65fb14440e3b9303f494d390b40` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.459479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b187f65fb14440e3b9303f494d390b40)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:45.461465 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:45.465430 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1305/2604] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:45.466317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6cc98-03f7-441c-9a76-7c0c78f09a74 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:45.467718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-17d6cc98-03f7-441c-9a76-7c0c78f09a74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:45.487964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6cc98-03f7-441c-9a76-7c0c78f09a74 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:45.497128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387136ef-8737-44a7-b018-e876e5464789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:45.498765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387136ef-8737-44a7-b018-e876e5464789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.498765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387136ef-8737-44a7-b018-e876e5464789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.499353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387136ef-8737-44a7-b018-e876e5464789 admin admin] PATH_INFO: `/v3/projects/b40dd686c68d462fbe7962e1ecb71c1a` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.501162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387136ef-8737-44a7-b018-e876e5464789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b40dd686c68d462fbe7962e1ecb71c1a)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:45.504650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387136ef-8737-44a7-b018-e876e5464789 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:45.544544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b187f65fb14440e3b9303f494d390b40', 'request_id': 'req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.552710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b187f65fb14440e3b9303f494d390b40', 'request_id': 'req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.562782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b187f65fb14440e3b9303f494d390b40', 'request_id': 'req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.567886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-387136ef-8737-44a7-b018-e876e5464789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b40dd686c68d462fbe7962e1ecb71c1a', 'request_id': 'req-387136ef-8737-44a7-b018-e876e5464789'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.576546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-387136ef-8737-44a7-b018-e876e5464789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b40dd686c68d462fbe7962e1ecb71c1a', 'request_id': 'req-387136ef-8737-44a7-b018-e876e5464789'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.577315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] The token cache is being invalidate because project b187f65fb14440e3b9303f494d390b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:45.577761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.578448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.579946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1300/2605] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:45 2022] DELETE /identity/v3/projects/b187f65fb14440e3b9303f494d390b40 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:45.580700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:45.581966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:45.585926 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-387136ef-8737-44a7-b018-e876e5464789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b40dd686c68d462fbe7962e1ecb71c1a', 'request_id': 'req-387136ef-8737-44a7-b018-e876e5464789'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.597442 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-387136ef-8737-44a7-b018-e876e5464789 admin admin] The token cache is being invalidate because project b40dd686c68d462fbe7962e1ecb71c1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:45.597956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-387136ef-8737-44a7-b018-e876e5464789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.598774 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-387136ef-8737-44a7-b018-e876e5464789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:45.600373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1306/2606] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:45 2022] DELETE /identity/v3/projects/b40dd686c68d462fbe7962e1ecb71c1a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:45.601705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5fd8d-330f-48dc-ba0b-b8ac33e7462d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.602070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5fd8d-330f-48dc-ba0b-b8ac33e7462d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.602424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5fd8d-330f-48dc-ba0b-b8ac33e7462d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.624921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-70b5fd8d-330f-48dc-ba0b-b8ac33e7462d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:45.626243 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-70b5fd8d-330f-48dc-ba0b-b8ac33e7462d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:45.652831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f4c70c-98b5-46f6-8227-e18fa1bcc5f9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:45.701132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1307/2607] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:45.709491 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c34b2-1580-4451-8ef8-b1173beee925 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.709741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c34b2-1580-4451-8ef8-b1173beee925 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.709990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099c34b2-1580-4451-8ef8-b1173beee925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.715061 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b394a4e-bd00-4dd3-b007-abcbb16d63c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:45.715771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b394a4e-bd00-4dd3-b007-abcbb16d63c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.716120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b394a4e-bd00-4dd3-b007-abcbb16d63c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.716512 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b394a4e-bd00-4dd3-b007-abcbb16d63c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.718735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2b394a4e-bd00-4dd3-b007-abcbb16d63c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:45.732928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-099c34b2-1580-4451-8ef8-b1173beee925 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:45.734291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-099c34b2-1580-4451-8ef8-b1173beee925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:45.742676 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1308/2608] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:45.747545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c34b2-1580-4451-8ef8-b1173beee925 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:45.757563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099c34b2-1580-4451-8ef8-b1173beee925 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:45.762886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b72c12-20ac-4655-bba5-db59fe5b9d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:45.763641 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b72c12-20ac-4655-bba5-db59fe5b9d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.764019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b72c12-20ac-4655-bba5-db59fe5b9d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.764301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b72c12-20ac-4655-bba5-db59fe5b9d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.764965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b72c12-20ac-4655-bba5-db59fe5b9d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:45.765276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b72c12-20ac-4655-bba5-db59fe5b9d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:45.767197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b72c12-20ac-4655-bba5-db59fe5b9d9c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:45.777272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:45.784647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1309/2609] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:30:45.785952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b72c12-20ac-4655-bba5-db59fe5b9d9c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:45.786140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:45.797861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b72c12-20ac-4655-bba5-db59fe5b9d9c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:45.805367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b394a4e-bd00-4dd3-b007-abcbb16d63c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:45.805625 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06688f4b-b1c7-4ceb-b6e5-9e3a4266a187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:45.806302 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06688f4b-b1c7-4ceb-b6e5-9e3a4266a187 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.806647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06688f4b-b1c7-4ceb-b6e5-9e3a4266a187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.806995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06688f4b-b1c7-4ceb-b6e5-9e3a4266a187 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.807303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b394a4e-bd00-4dd3-b007-abcbb16d63c5 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:45.807765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06688f4b-b1c7-4ceb-b6e5-9e3a4266a187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:45.810002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06688f4b-b1c7-4ceb-b6e5-9e3a4266a187 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:45.818303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1301/2610] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:45.850112 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1310/2611] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:45 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:45.850482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b394a4e-bd00-4dd3-b007-abcbb16d63c5 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:45.860293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b394a4e-bd00-4dd3-b007-abcbb16d63c5 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:45.922663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef742f-fba7-4f34-8d90-91abbc517389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:45.923266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef742f-fba7-4f34-8d90-91abbc517389 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.923542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef742f-fba7-4f34-8d90-91abbc517389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.923811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efef742f-fba7-4f34-8d90-91abbc517389 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.924436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef742f-fba7-4f34-8d90-91abbc517389 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:45.926515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efef742f-fba7-4f34-8d90-91abbc517389 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:45.951861 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1302/2612] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:45 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:45.953246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06688f4b-b1c7-4ceb-b6e5-9e3a4266a187 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:45.963842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06688f4b-b1c7-4ceb-b6e5-9e3a4266a187 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:45.969194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b214aa7-822f-4013-a604-5aa6de6bf722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:45.969873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b214aa7-822f-4013-a604-5aa6de6bf722 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:45.970258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b214aa7-822f-4013-a604-5aa6de6bf722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:45.970605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b214aa7-822f-4013-a604-5aa6de6bf722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:45.971323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b214aa7-822f-4013-a604-5aa6de6bf722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:45.973010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b214aa7-822f-4013-a604-5aa6de6bf722 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:45.983707 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1311/2613] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:45.988784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef742f-fba7-4f34-8d90-91abbc517389 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:45.999340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efef742f-fba7-4f34-8d90-91abbc517389 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:46.004412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed7860-05c9-41b4-b9d8-e785dbd493fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:46.005606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed7860-05c9-41b4-b9d8-e785dbd493fe admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:46.005606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed7860-05c9-41b4-b9d8-e785dbd493fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:46.005606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed7860-05c9-41b4-b9d8-e785dbd493fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:46.006201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed7860-05c9-41b4-b9d8-e785dbd493fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:46.007796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed7860-05c9-41b4-b9d8-e785dbd493fe admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:46.022889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1303/2614] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:46.024793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b214aa7-822f-4013-a604-5aa6de6bf722 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:46.036618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b214aa7-822f-4013-a604-5aa6de6bf722 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:46.041639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b7476-3e82-4bb6-b737-091280f51ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:46.042364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b7476-3e82-4bb6-b737-091280f51ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:46.042656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b7476-3e82-4bb6-b737-091280f51ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:46.042934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b7476-3e82-4bb6-b737-091280f51ba9 admin admin] PATH_INFO: `/v3/projects/44f9cb40883246dab0f86453774cc817/users/a9a98cfc8d834c858bae307c8521b278/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:46.060360 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b7476-3e82-4bb6-b737-091280f51ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44f9cb40883246dab0f86453774cc817, user_id=a9a98cfc8d834c858bae307c8521b278, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:46.063472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b7476-3e82-4bb6-b737-091280f51ba9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:46.078801 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1312/2615] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:46 2022] PUT /identity/v3/projects/44f9cb40883246dab0f86453774cc817/users/a9a98cfc8d834c858bae307c8521b278/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:46.091218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77748c3d-8e3f-4d5a-a254-3fd25e000bb6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:46.091528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77748c3d-8e3f-4d5a-a254-3fd25e000bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:46.091850 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77748c3d-8e3f-4d5a-a254-3fd25e000bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:46.141980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-77748c3d-8e3f-4d5a-a254-3fd25e000bb6 None None] MFA Rules not processed for user `a9a98cfc8d834c858bae307c8521b278`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:46.144170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-77748c3d-8e3f-4d5a-a254-3fd25e000bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:46.224405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1304/2616] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:46.229446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b7476-3e82-4bb6-b737-091280f51ba9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:46.302825 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b7476-3e82-4bb6-b737-091280f51ba9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:46.307439 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c36a58-85a7-429c-b9f3-cd468926cc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:46.308560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c36a58-85a7-429c-b9f3-cd468926cc6c admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:46.308838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c36a58-85a7-429c-b9f3-cd468926cc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:46.309121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c36a58-85a7-429c-b9f3-cd468926cc6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:46.309745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c36a58-85a7-429c-b9f3-cd468926cc6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:46.311678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c36a58-85a7-429c-b9f3-cd468926cc6c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:46.339753 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1313/2617] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:46 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:46.344234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77748c3d-8e3f-4d5a-a254-3fd25e000bb6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:46.356420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77748c3d-8e3f-4d5a-a254-3fd25e000bb6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:46.410418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:46.418098 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22a27d-2be1-4ac6-a557-b90a2efc31aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:46.419062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22a27d-2be1-4ac6-a557-b90a2efc31aa admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:46.419550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22a27d-2be1-4ac6-a557-b90a2efc31aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:46.420015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b22a27d-2be1-4ac6-a557-b90a2efc31aa admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:46.420948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22a27d-2be1-4ac6-a557-b90a2efc31aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:46.422938 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b22a27d-2be1-4ac6-a557-b90a2efc31aa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:46.450257 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1305/2618] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:46 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:46.458756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c36a58-85a7-429c-b9f3-cd468926cc6c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:46.469355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c36a58-85a7-429c-b9f3-cd468926cc6c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:46.474132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a51d-86ba-4a54-8027-0474af6cd827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:46.474738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a51d-86ba-4a54-8027-0474af6cd827 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:46.475016 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a51d-86ba-4a54-8027-0474af6cd827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:46.475283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5a51d-86ba-4a54-8027-0474af6cd827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:46.475994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a51d-86ba-4a54-8027-0474af6cd827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:46.477585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5a51d-86ba-4a54-8027-0474af6cd827 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:46.489441 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1314/2619] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:46.493670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22a27d-2be1-4ac6-a557-b90a2efc31aa admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:46.504597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b22a27d-2be1-4ac6-a557-b90a2efc31aa admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:46.509327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4bd39-8dcf-4bf9-855a-0093759a8d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:46.509990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4bd39-8dcf-4bf9-855a-0093759a8d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:46.510295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4bd39-8dcf-4bf9-855a-0093759a8d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:46.510566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4bd39-8dcf-4bf9-855a-0093759a8d68 admin admin] PATH_INFO: `/v3/projects/5d63d6b6b16746feb5ac03bdc8a781bb/users/fde941c3a42f4df1a651a8381945e19f/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:46.527208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4bd39-8dcf-4bf9-855a-0093759a8d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d63d6b6b16746feb5ac03bdc8a781bb, user_id=fde941c3a42f4df1a651a8381945e19f, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:46.529860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4bd39-8dcf-4bf9-855a-0093759a8d68 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:46.545114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1306/2620] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:46 2022] PUT /identity/v3/projects/5d63d6b6b16746feb5ac03bdc8a781bb/users/fde941c3a42f4df1a651a8381945e19f/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:46.549357 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a51d-86ba-4a54-8027-0474af6cd827 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:46.560900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5a51d-86ba-4a54-8027-0474af6cd827 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:46.619137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fecdb7b-d845-404c-8ee7-41808a2aafb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:46.620003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecdb7b-d845-404c-8ee7-41808a2aafb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:46.620140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecdb7b-d845-404c-8ee7-41808a2aafb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:46.620364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fecdb7b-d845-404c-8ee7-41808a2aafb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:46.621049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fecdb7b-d845-404c-8ee7-41808a2aafb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:46.622697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fecdb7b-d845-404c-8ee7-41808a2aafb5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:46.632945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1315/2621] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:46.637343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4bd39-8dcf-4bf9-855a-0093759a8d68 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:46.649154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4bd39-8dcf-4bf9-855a-0093759a8d68 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:46.654581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646c883-bfe2-494a-b2aa-e9cf92f74d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:46.655240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646c883-bfe2-494a-b2aa-e9cf92f74d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:46.655513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646c883-bfe2-494a-b2aa-e9cf92f74d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:46.655781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646c883-bfe2-494a-b2aa-e9cf92f74d69 admin admin] PATH_INFO: `/v3/projects/5d63d6b6b16746feb5ac03bdc8a781bb/users/fde941c3a42f4df1a651a8381945e19f/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:46.657684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646c883-bfe2-494a-b2aa-e9cf92f74d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d63d6b6b16746feb5ac03bdc8a781bb, user_id=fde941c3a42f4df1a651a8381945e19f, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:46.660305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646c883-bfe2-494a-b2aa-e9cf92f74d69 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:46.671677 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1307/2622] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:46 2022] PUT /identity/v3/projects/5d63d6b6b16746feb5ac03bdc8a781bb/users/fde941c3a42f4df1a651a8381945e19f/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:46.681820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6da5a-ceb0-4083-9208-a23d4da647ea None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:46.682148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6da5a-ceb0-4083-9208-a23d4da647ea None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:46.682430 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6da5a-ceb0-4083-9208-a23d4da647ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:46.708954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:46.712613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-8cd6da5a-ceb0-4083-9208-a23d4da647ea None None] MFA Rules not processed for user `fde941c3a42f4df1a651a8381945e19f`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:46.715265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8cd6da5a-ceb0-4083-9208-a23d4da647ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:46.789573 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1316/2623] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:46.801910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646c883-bfe2-494a-b2aa-e9cf92f74d69 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:46.803836 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f646c883-bfe2-494a-b2aa-e9cf92f74d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:46.865514 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646c883-bfe2-494a-b2aa-e9cf92f74d69 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:46.869234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e96ee-f4cd-4f1b-950a-944b7b4a6ae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:46.869851 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e96ee-f4cd-4f1b-950a-944b7b4a6ae4 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:46.870156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e96ee-f4cd-4f1b-950a-944b7b4a6ae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:46.871213 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e96ee-f4cd-4f1b-950a-944b7b4a6ae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:46.880593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e96ee-f4cd-4f1b-950a-944b7b4a6ae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:46.882038 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e96ee-f4cd-4f1b-950a-944b7b4a6ae4 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:46.892992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1308/2624] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:47.478201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6da5a-ceb0-4083-9208-a23d4da647ea None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:47.550355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6da5a-ceb0-4083-9208-a23d4da647ea None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:47.555527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:47.556007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:47.559487 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:47.559487 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] PATH_INFO: `/v3/users/a9a98cfc8d834c858bae307c8521b278` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:47.559487 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a98cfc8d834c858bae307c8521b278)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:47.559772 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:47.610741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a98cfc8d834c858bae307c8521b278'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:47.619038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a98cfc8d834c858bae307c8521b278'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:47.619729 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a98cfc8d834c858bae307c8521b278'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:47.628014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a98cfc8d834c858bae307c8521b278'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:47.641095 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a98cfc8d834c858bae307c8521b278'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:47.647470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1317/2625] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:47 2022] DELETE /identity/v3/users/a9a98cfc8d834c858bae307c8521b278 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:47.653695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aea063-553c-43ac-9293-1e32ff009726 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:47.654086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aea063-553c-43ac-9293-1e32ff009726 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:47.654703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aea063-553c-43ac-9293-1e32ff009726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:47.679326 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e3aea063-553c-43ac-9293-1e32ff009726 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:47.680442 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e3aea063-553c-43ac-9293-1e32ff009726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:47.737984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1309/2626] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:47.748110 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:47.749771 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:47.773111 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:47.816162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c4570d-8877-4f08-a5e2-62b8eef9f5d9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:47.820284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7303a65-b2dd-4ac4-b088-6ffee964d77e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:47.820936 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7303a65-b2dd-4ac4-b088-6ffee964d77e service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:47.821280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7303a65-b2dd-4ac4-b088-6ffee964d77e service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:47.821658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7303a65-b2dd-4ac4-b088-6ffee964d77e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:47.832603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7303a65-b2dd-4ac4-b088-6ffee964d77e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:47.834350 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7303a65-b2dd-4ac4-b088-6ffee964d77e service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:47.845480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1318/2627] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:48.117310 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aea063-553c-43ac-9293-1e32ff009726 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:48.119107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e3aea063-553c-43ac-9293-1e32ff009726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:48.132625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3aea063-553c-43ac-9293-1e32ff009726 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:48.137988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:48.138658 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:48.138938 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:48.139247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] PATH_INFO: `/v3/projects/44f9cb40883246dab0f86453774cc817` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:48.147049 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] RBAC: Authorizing `identity:delete_project(project_id=44f9cb40883246dab0f86453774cc817)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:48.148985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:48.202790 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:48.204351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f9cb40883246dab0f86453774cc817', 'request_id': 'req-6ce22147-25ff-4819-8dee-021d8e1cc39f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.212739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f9cb40883246dab0f86453774cc817', 'request_id': 'req-6ce22147-25ff-4819-8dee-021d8e1cc39f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.218879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f9cb40883246dab0f86453774cc817', 'request_id': 'req-6ce22147-25ff-4819-8dee-021d8e1cc39f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.231015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] The token cache is being invalidate because project 44f9cb40883246dab0f86453774cc817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:48.231303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.232045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.233520 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1310/2628] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:48 2022] DELETE /identity/v3/projects/44f9cb40883246dab0f86453774cc817 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:48.237543 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7303a65-b2dd-4ac4-b088-6ffee964d77e service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:48.249637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7303a65-b2dd-4ac4-b088-6ffee964d77e service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:48.310028 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:48.310683 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:48.310966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:48.311243 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] PATH_INFO: `/v3/users/fde941c3a42f4df1a651a8381945e19f` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:48.312372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fde941c3a42f4df1a651a8381945e19f)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:48.314321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:48.361261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fde941c3a42f4df1a651a8381945e19f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.369746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fde941c3a42f4df1a651a8381945e19f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.371011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fde941c3a42f4df1a651a8381945e19f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.378601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fde941c3a42f4df1a651a8381945e19f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.390951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fde941c3a42f4df1a651a8381945e19f'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.400562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1319/2629] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:48 2022] DELETE /identity/v3/users/fde941c3a42f4df1a651a8381945e19f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:48.667172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:48.669801 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:48.685353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce22147-25ff-4819-8dee-021d8e1cc39f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:48.744922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914cea6-7ea1-458a-8f7c-42469f78408c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:48.745534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914cea6-7ea1-458a-8f7c-42469f78408c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:48.745893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914cea6-7ea1-458a-8f7c-42469f78408c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:48.746234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914cea6-7ea1-458a-8f7c-42469f78408c admin admin] PATH_INFO: `/v3/projects/5d63d6b6b16746feb5ac03bdc8a781bb` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:48.747335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914cea6-7ea1-458a-8f7c-42469f78408c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d63d6b6b16746feb5ac03bdc8a781bb)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:48.749234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914cea6-7ea1-458a-8f7c-42469f78408c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:48.769996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:48.771543 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:48.785579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbc0fb-14f0-4454-9a27-e10a94122089 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:48.790240 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11d9c7-ac92-4655-b464-738f41a78d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:48.790873 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11d9c7-ac92-4655-b464-738f41a78d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:48.791150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11d9c7-ac92-4655-b464-738f41a78d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:48.791423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11d9c7-ac92-4655-b464-738f41a78d56 admin admin] PATH_INFO: `/v3/users/38a8024cdd55440e83e3a521621ed6dd` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:48.793169 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11d9c7-ac92-4655-b464-738f41a78d56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38a8024cdd55440e83e3a521621ed6dd)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:48.795114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11d9c7-ac92-4655-b464-738f41a78d56 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:48.803802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7914cea6-7ea1-458a-8f7c-42469f78408c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d63d6b6b16746feb5ac03bdc8a781bb', 'request_id': 'req-7914cea6-7ea1-458a-8f7c-42469f78408c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.811494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7914cea6-7ea1-458a-8f7c-42469f78408c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d63d6b6b16746feb5ac03bdc8a781bb', 'request_id': 'req-7914cea6-7ea1-458a-8f7c-42469f78408c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.817573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7914cea6-7ea1-458a-8f7c-42469f78408c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d63d6b6b16746feb5ac03bdc8a781bb', 'request_id': 'req-7914cea6-7ea1-458a-8f7c-42469f78408c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.830597 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7914cea6-7ea1-458a-8f7c-42469f78408c admin admin] The token cache is being invalidate because project 5d63d6b6b16746feb5ac03bdc8a781bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:48.830887 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7914cea6-7ea1-458a-8f7c-42469f78408c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.831540 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7914cea6-7ea1-458a-8f7c-42469f78408c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.833035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1311/2630] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:48 2022] DELETE /identity/v3/projects/5d63d6b6b16746feb5ac03bdc8a781bb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:48.834655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:48.837825 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb8a6f-9a39-4039-bf22-55d6e47b4ac8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:48.838141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb8a6f-9a39-4039-bf22-55d6e47b4ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:48.838418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb8a6f-9a39-4039-bf22-55d6e47b4ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:48.847938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3d11d9c7-ac92-4655-b464-738f41a78d56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38a8024cdd55440e83e3a521621ed6dd'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.855973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3d11d9c7-ac92-4655-b464-738f41a78d56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38a8024cdd55440e83e3a521621ed6dd'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.856685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3d11d9c7-ac92-4655-b464-738f41a78d56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38a8024cdd55440e83e3a521621ed6dd'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.861063 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6ccb8a6f-9a39-4039-bf22-55d6e47b4ac8 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:48.864587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6ccb8a6f-9a39-4039-bf22-55d6e47b4ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:48.865438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3d11d9c7-ac92-4655-b464-738f41a78d56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38a8024cdd55440e83e3a521621ed6dd'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.878130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3d11d9c7-ac92-4655-b464-738f41a78d56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38a8024cdd55440e83e3a521621ed6dd'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:48.884200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1320/2631] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:48 2022] DELETE /identity/v3/users/38a8024cdd55440e83e3a521621ed6dd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:48.889267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20700fe1-2c34-43ed-9e9c-bb27e9740cc0 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:48.889558 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20700fe1-2c34-43ed-9e9c-bb27e9740cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:48.889827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20700fe1-2c34-43ed-9e9c-bb27e9740cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:48.911860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-20700fe1-2c34-43ed-9e9c-bb27e9740cc0 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:48.913066 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-20700fe1-2c34-43ed-9e9c-bb27e9740cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:48.924971 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1312/2632] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:48.976532 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1321/2633] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:30:48.986901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb8a6f-9a39-4039-bf22-55d6e47b4ac8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:48.988548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6ccb8a6f-9a39-4039-bf22-55d6e47b4ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:48.992612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a12f6c-45e8-4656-bf61-b44cb5b0e542 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:48.992904 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a12f6c-45e8-4656-bf61-b44cb5b0e542 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:48.993217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a12f6c-45e8-4656-bf61-b44cb5b0e542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:48.999909 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:49.012742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f3a12f6c-45e8-4656-bf61-b44cb5b0e542 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:49.013995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f3a12f6c-45e8-4656-bf61-b44cb5b0e542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:49.021814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1322/2634] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:30:49.025912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a12f6c-45e8-4656-bf61-b44cb5b0e542 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.035447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a12f6c-45e8-4656-bf61-b44cb5b0e542 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.040130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480fa3-2940-4d35-9cd4-ba39157aa357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.040728 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480fa3-2940-4d35-9cd4-ba39157aa357 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.041010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480fa3-2940-4d35-9cd4-ba39157aa357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.041284 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff480fa3-2940-4d35-9cd4-ba39157aa357 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.041925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff480fa3-2940-4d35-9cd4-ba39157aa357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:49.042291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff480fa3-2940-4d35-9cd4-ba39157aa357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.043883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff480fa3-2940-4d35-9cd4-ba39157aa357 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.053092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb8a6f-9a39-4039-bf22-55d6e47b4ac8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.055545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1323/2635] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 21 16:30:49.059764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480fa3-2940-4d35-9cd4-ba39157aa357 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.069388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff480fa3-2940-4d35-9cd4-ba39157aa357 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.074551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fedc5b-871d-4cb8-a93d-d59af03a60f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.075142 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fedc5b-871d-4cb8-a93d-d59af03a60f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.075426 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fedc5b-871d-4cb8-a93d-d59af03a60f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.075700 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fedc5b-871d-4cb8-a93d-d59af03a60f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.076346 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fedc5b-871d-4cb8-a93d-d59af03a60f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.078244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fedc5b-871d-4cb8-a93d-d59af03a60f9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.105680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e7818-3ac2-424f-9bee-56a54b97c836 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.106249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1324/2636] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:49 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:49.106418 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e7818-3ac2-424f-9bee-56a54b97c836 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.106586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e7818-3ac2-424f-9bee-56a54b97c836 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.106835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e7818-3ac2-424f-9bee-56a54b97c836 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.110267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fedc5b-871d-4cb8-a93d-d59af03a60f9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.116170 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e7818-3ac2-424f-9bee-56a54b97c836 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.117600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e7818-3ac2-424f-9bee-56a54b97c836 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.127242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1313/2637] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:49.177716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fedc5b-871d-4cb8-a93d-d59af03a60f9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.183030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a8339-bdc2-4de0-b887-6666bc1941e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.183675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a8339-bdc2-4de0-b887-6666bc1941e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.183987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a8339-bdc2-4de0-b887-6666bc1941e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.184262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a8339-bdc2-4de0-b887-6666bc1941e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.185004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a8339-bdc2-4de0-b887-6666bc1941e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.187012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335a8339-bdc2-4de0-b887-6666bc1941e8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.210210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:49.219747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1325/2638] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:49 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:49.222494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e7818-3ac2-424f-9bee-56a54b97c836 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.235441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e7818-3ac2-424f-9bee-56a54b97c836 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.240482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05294ed-1605-411f-aa60-93c1dbc27554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.241239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05294ed-1605-411f-aa60-93c1dbc27554 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.241577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05294ed-1605-411f-aa60-93c1dbc27554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.241850 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05294ed-1605-411f-aa60-93c1dbc27554 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.242544 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05294ed-1605-411f-aa60-93c1dbc27554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.244139 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05294ed-1605-411f-aa60-93c1dbc27554 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.256942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1314/2639] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:49.261386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a8339-bdc2-4de0-b887-6666bc1941e8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.272437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335a8339-bdc2-4de0-b887-6666bc1941e8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.278323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e3da3-4559-40aa-bf06-f90429785481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.278952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e3da3-4559-40aa-bf06-f90429785481 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.279230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e3da3-4559-40aa-bf06-f90429785481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.279508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e3da3-4559-40aa-bf06-f90429785481 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.280164 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e3da3-4559-40aa-bf06-f90429785481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.281757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e3da3-4559-40aa-bf06-f90429785481 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.293373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1326/2640] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:49.298166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05294ed-1605-411f-aa60-93c1dbc27554 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.317179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05294ed-1605-411f-aa60-93c1dbc27554 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.324428 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d15669-c53d-4f8d-a89b-d7b6a8e6ce5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.325305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d15669-c53d-4f8d-a89b-d7b6a8e6ce5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.325587 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d15669-c53d-4f8d-a89b-d7b6a8e6ce5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.325860 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d15669-c53d-4f8d-a89b-d7b6a8e6ce5c admin admin] PATH_INFO: `/v3/projects/b00bbf6ff5d24c98a69f4cc78fd82051/users/56f5ca121eeb4c29a2c85a02078c8b2c/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.345432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d15669-c53d-4f8d-a89b-d7b6a8e6ce5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00bbf6ff5d24c98a69f4cc78fd82051, user_id=56f5ca121eeb4c29a2c85a02078c8b2c, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.348409 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d15669-c53d-4f8d-a89b-d7b6a8e6ce5c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.373360 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1315/2641] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:49 2022] PUT /identity/v3/projects/b00bbf6ff5d24c98a69f4cc78fd82051/users/56f5ca121eeb4c29a2c85a02078c8b2c/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:49.383612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d50e09-cc29-4652-bf7e-6c4fb32f8da4 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.383897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d50e09-cc29-4652-bf7e-6c4fb32f8da4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.384169 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d50e09-cc29-4652-bf7e-6c4fb32f8da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.424685 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c9d50e09-cc29-4652-bf7e-6c4fb32f8da4 None None] MFA Rules not processed for user `56f5ca121eeb4c29a2c85a02078c8b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:49.426100 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c9d50e09-cc29-4652-bf7e-6c4fb32f8da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:49.447398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d15669-c53d-4f8d-a89b-d7b6a8e6ce5c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.449426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c2d15669-c53d-4f8d-a89b-d7b6a8e6ce5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:49.516852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1327/2642] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:49.521502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d50e09-cc29-4652-bf7e-6c4fb32f8da4 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.536804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d15669-c53d-4f8d-a89b-d7b6a8e6ce5c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.543620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.544254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.544531 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.544801 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] PATH_INFO: `/v3/projects/760e8a3fbe3b4b6da8312017eb7af2b0` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.545926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=760e8a3fbe3b4b6da8312017eb7af2b0)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.547929 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.588044 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d50e09-cc29-4652-bf7e-6c4fb32f8da4 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.590864 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:49.605467 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '760e8a3fbe3b4b6da8312017eb7af2b0', 'request_id': 'req-d59a57b2-e5c3-4163-97cb-603cf36ad628'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:49.613233 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '760e8a3fbe3b4b6da8312017eb7af2b0', 'request_id': 'req-d59a57b2-e5c3-4163-97cb-603cf36ad628'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:49.619131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '760e8a3fbe3b4b6da8312017eb7af2b0', 'request_id': 'req-d59a57b2-e5c3-4163-97cb-603cf36ad628'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:49.630354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] The token cache is being invalidate because project 760e8a3fbe3b4b6da8312017eb7af2b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:49.630640 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:49.631304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:49.633066 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1316/2643] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:49 2022] DELETE /identity/v3/projects/760e8a3fbe3b4b6da8312017eb7af2b0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:49.636568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.637875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:49.649018 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea7735-c570-4ab5-bddf-b1766c3e93a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.649628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea7735-c570-4ab5-bddf-b1766c3e93a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.649929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea7735-c570-4ab5-bddf-b1766c3e93a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.650266 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea7735-c570-4ab5-bddf-b1766c3e93a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.650949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea7735-c570-4ab5-bddf-b1766c3e93a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.653422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea7735-c570-4ab5-bddf-b1766c3e93a8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.684816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1328/2644] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:49 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:49.685819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea7735-c570-4ab5-bddf-b1766c3e93a8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.687459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-48ea7735-c570-4ab5-bddf-b1766c3e93a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:49.700906 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:49.708457 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a57b2-e5c3-4163-97cb-603cf36ad628 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.755256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea7735-c570-4ab5-bddf-b1766c3e93a8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.760179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ccfb5-722f-4b22-9526-46dc9e572ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.760943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ccfb5-722f-4b22-9526-46dc9e572ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.761226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ccfb5-722f-4b22-9526-46dc9e572ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.761509 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ccfb5-722f-4b22-9526-46dc9e572ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.762191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ccfb5-722f-4b22-9526-46dc9e572ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.764115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ccfb5-722f-4b22-9526-46dc9e572ff0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.767898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.768506 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.768784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.769054 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] PATH_INFO: `/v3/users/ccea7d846ad342d6b86f10a3c372ac9f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.770208 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccea7d846ad342d6b86f10a3c372ac9f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.772091 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.785601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1329/2645] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:49 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:49.789345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ccfb5-722f-4b22-9526-46dc9e572ff0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.801303 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ccfb5-722f-4b22-9526-46dc9e572ff0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.807597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2acb8e-06b6-4495-8a90-56d7516fffef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.808815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2acb8e-06b6-4495-8a90-56d7516fffef admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.809088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2acb8e-06b6-4495-8a90-56d7516fffef admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.809357 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2acb8e-06b6-4495-8a90-56d7516fffef admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.809999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2acb8e-06b6-4495-8a90-56d7516fffef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.811612 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2acb8e-06b6-4495-8a90-56d7516fffef admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.822190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1330/2646] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:49.826478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2acb8e-06b6-4495-8a90-56d7516fffef admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.828038 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccea7d846ad342d6b86f10a3c372ac9f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:49.835754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccea7d846ad342d6b86f10a3c372ac9f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:49.836133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2acb8e-06b6-4495-8a90-56d7516fffef admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:49.840817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fbe5e-0034-45de-a67d-95833d05af7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:49.841470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fbe5e-0034-45de-a67d-95833d05af7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:49.841756 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fbe5e-0034-45de-a67d-95833d05af7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:49.842029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fbe5e-0034-45de-a67d-95833d05af7d admin admin] PATH_INFO: `/v3/projects/40169ef9f8e24f75a92fbc2e07f61b76/users/29399e1aa91549ac93d5cc0f4ef88a7d/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:49.848535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:49.850261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccea7d846ad342d6b86f10a3c372ac9f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:49.851075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccea7d846ad342d6b86f10a3c372ac9f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:49.858917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fbe5e-0034-45de-a67d-95833d05af7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=40169ef9f8e24f75a92fbc2e07f61b76, user_id=29399e1aa91549ac93d5cc0f4ef88a7d, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:49.861598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2fbe5e-0034-45de-a67d-95833d05af7d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:49.861794 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccea7d846ad342d6b86f10a3c372ac9f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:49.868915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1317/2647] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:49 2022] DELETE /identity/v3/users/ccea7d846ad342d6b86f10a3c372ac9f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:49.875731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1331/2648] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:49 2022] PUT /identity/v3/projects/40169ef9f8e24f75a92fbc2e07f61b76/users/29399e1aa91549ac93d5cc0f4ef88a7d/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:49.879626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:49.881090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:49.952884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757006a9-a650-4943-9df2-4349a7de6efe admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:50.014765 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c0022-6f96-4716-8635-9ccd905cefc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:50.015637 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c0022-6f96-4716-8635-9ccd905cefc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:50.015919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c0022-6f96-4716-8635-9ccd905cefc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:50.016196 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c0022-6f96-4716-8635-9ccd905cefc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:50.016841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c0022-6f96-4716-8635-9ccd905cefc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:50.019877 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c0022-6f96-4716-8635-9ccd905cefc8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:50.030340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1318/2649] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:50.033746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fbe5e-0034-45de-a67d-95833d05af7d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:50.044763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2fbe5e-0034-45de-a67d-95833d05af7d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:50.049611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41fe04-9e9a-4992-970c-e803e35ea8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:50.050480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41fe04-9e9a-4992-970c-e803e35ea8e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:50.050765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41fe04-9e9a-4992-970c-e803e35ea8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:50.051042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41fe04-9e9a-4992-970c-e803e35ea8e0 admin admin] PATH_INFO: `/v3/projects/40169ef9f8e24f75a92fbc2e07f61b76/users/29399e1aa91549ac93d5cc0f4ef88a7d/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:50.052986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41fe04-9e9a-4992-970c-e803e35ea8e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40169ef9f8e24f75a92fbc2e07f61b76, user_id=29399e1aa91549ac93d5cc0f4ef88a7d, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:50.055674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a41fe04-9e9a-4992-970c-e803e35ea8e0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:50.067645 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1332/2650] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:50 2022] PUT /identity/v3/projects/40169ef9f8e24f75a92fbc2e07f61b76/users/29399e1aa91549ac93d5cc0f4ef88a7d/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:50.079031 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea32ea-0fbf-4a78-93c6-9154e9bbcb85 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:50.079319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea32ea-0fbf-4a78-93c6-9154e9bbcb85 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:50.079593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea32ea-0fbf-4a78-93c6-9154e9bbcb85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:50.108304 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9bea32ea-0fbf-4a78-93c6-9154e9bbcb85 None None] MFA Rules not processed for user `29399e1aa91549ac93d5cc0f4ef88a7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:50.109526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9bea32ea-0fbf-4a78-93c6-9154e9bbcb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:50.150582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41fe04-9e9a-4992-970c-e803e35ea8e0 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:50.152962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9a41fe04-9e9a-4992-970c-e803e35ea8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:50.194481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1319/2651] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:50.208353 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea32ea-0fbf-4a78-93c6-9154e9bbcb85 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:50.209834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9bea32ea-0fbf-4a78-93c6-9154e9bbcb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:50.228429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:50.236310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41fe04-9e9a-4992-970c-e803e35ea8e0 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:50.241331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:50.241962 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:50.242275 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:50.242547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] PATH_INFO: `/v3/projects/ccfd93f5d8274c069ff1250f669933a1` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:50.243706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccfd93f5d8274c069ff1250f669933a1)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:50.247050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:50.276327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea32ea-0fbf-4a78-93c6-9154e9bbcb85 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:50.280290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434aacf8-b932-47d9-9574-06f0c99c09ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:50.280913 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434aacf8-b932-47d9-9574-06f0c99c09ed service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:50.281175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434aacf8-b932-47d9-9574-06f0c99c09ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:50.281446 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434aacf8-b932-47d9-9574-06f0c99c09ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:50.283309 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-434aacf8-b932-47d9-9574-06f0c99c09ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:50.294429 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccfd93f5d8274c069ff1250f669933a1', 'request_id': 'req-292f4e75-d607-47ff-aecf-3ceb24715acf'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:50.298345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:50.302901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccfd93f5d8274c069ff1250f669933a1', 'request_id': 'req-292f4e75-d607-47ff-aecf-3ceb24715acf'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:50.311921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccfd93f5d8274c069ff1250f669933a1', 'request_id': 'req-292f4e75-d607-47ff-aecf-3ceb24715acf'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:50.323981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] The token cache is being invalidate because project ccfd93f5d8274c069ff1250f669933a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:50.324287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:50.324971 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:50.326513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1333/2652] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:50 2022] DELETE /identity/v3/projects/ccfd93f5d8274c069ff1250f669933a1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:50.330277 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:50.331591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:50.345514 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434aacf8-b932-47d9-9574-06f0c99c09ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:50.346998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434aacf8-b932-47d9-9574-06f0c99c09ed service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:50.356040 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1320/2653] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:30:50.403323 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f4e75-d607-47ff-aecf-3ceb24715acf admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:50.408280 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a318247-1150-42de-a2ad-7ec96a6cca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:50.408914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a318247-1150-42de-a2ad-7ec96a6cca70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:50.409196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a318247-1150-42de-a2ad-7ec96a6cca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:50.409470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a318247-1150-42de-a2ad-7ec96a6cca70 admin admin] PATH_INFO: `/v3/users/8b2bf77df28b494586da64d63fe18c74` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:50.410638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a318247-1150-42de-a2ad-7ec96a6cca70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2bf77df28b494586da64d63fe18c74)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:50.412577 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a318247-1150-42de-a2ad-7ec96a6cca70 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:50.462272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6a318247-1150-42de-a2ad-7ec96a6cca70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2bf77df28b494586da64d63fe18c74'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:50.470769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6a318247-1150-42de-a2ad-7ec96a6cca70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2bf77df28b494586da64d63fe18c74'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:50.471485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6a318247-1150-42de-a2ad-7ec96a6cca70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2bf77df28b494586da64d63fe18c74'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:50.480512 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6a318247-1150-42de-a2ad-7ec96a6cca70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2bf77df28b494586da64d63fe18c74'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:50.496576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-6a318247-1150-42de-a2ad-7ec96a6cca70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2bf77df28b494586da64d63fe18c74'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:50.503636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1334/2654] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:50 2022] DELETE /identity/v3/users/8b2bf77df28b494586da64d63fe18c74 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:50.844894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434aacf8-b932-47d9-9574-06f0c99c09ed service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:50.857240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434aacf8-b932-47d9-9574-06f0c99c09ed service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:50.920702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16326d98-fd92-4827-936b-be28b83817c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:50.921338 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16326d98-fd92-4827-936b-be28b83817c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:50.921618 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16326d98-fd92-4827-936b-be28b83817c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:50.921892 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16326d98-fd92-4827-936b-be28b83817c3 admin admin] PATH_INFO: `/v3/projects/2cd56e174bb941e0b5dc91d099d3d812` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:50.923089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16326d98-fd92-4827-936b-be28b83817c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cd56e174bb941e0b5dc91d099d3d812)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:50.925003 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16326d98-fd92-4827-936b-be28b83817c3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:50.974569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-16326d98-fd92-4827-936b-be28b83817c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cd56e174bb941e0b5dc91d099d3d812', 'request_id': 'req-16326d98-fd92-4827-936b-be28b83817c3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:50.983114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-16326d98-fd92-4827-936b-be28b83817c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cd56e174bb941e0b5dc91d099d3d812', 'request_id': 'req-16326d98-fd92-4827-936b-be28b83817c3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:50.991982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-16326d98-fd92-4827-936b-be28b83817c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cd56e174bb941e0b5dc91d099d3d812', 'request_id': 'req-16326d98-fd92-4827-936b-be28b83817c3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:51.002273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:51.007907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-16326d98-fd92-4827-936b-be28b83817c3 admin admin] The token cache is being invalidate because project 2cd56e174bb941e0b5dc91d099d3d812 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:51.008305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-16326d98-fd92-4827-936b-be28b83817c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:51.014635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-16326d98-fd92-4827-936b-be28b83817c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:51.018007 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1321/2655] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:50 2022] DELETE /identity/v3/projects/2cd56e174bb941e0b5dc91d099d3d812 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:51.025145 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086ef40-dc02-431d-9e6a-56d64df818f9 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.025958 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086ef40-dc02-431d-9e6a-56d64df818f9 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.025958 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086ef40-dc02-431d-9e6a-56d64df818f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.048573 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-5086ef40-dc02-431d-9e6a-56d64df818f9 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:51.049786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5086ef40-dc02-431d-9e6a-56d64df818f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:51.112826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1335/2656] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:51.122489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4828da-88c2-4fa7-915b-38cd4924e9cd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.122767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4828da-88c2-4fa7-915b-38cd4924e9cd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.123044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4828da-88c2-4fa7-915b-38cd4924e9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.144380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1e4828da-88c2-4fa7-915b-38cd4924e9cd None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:51.145649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1e4828da-88c2-4fa7-915b-38cd4924e9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:51.154826 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1322/2657] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:51.160851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086ef40-dc02-431d-9e6a-56d64df818f9 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:51.166238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:51.172983 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086ef40-dc02-431d-9e6a-56d64df818f9 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:51.178872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4781e41-42c0-4396-9f37-be2bc9579268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:51.179492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4781e41-42c0-4396-9f37-be2bc9579268 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.179791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4781e41-42c0-4396-9f37-be2bc9579268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.180044 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4781e41-42c0-4396-9f37-be2bc9579268 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.180695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4781e41-42c0-4396-9f37-be2bc9579268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:51.180998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4781e41-42c0-4396-9f37-be2bc9579268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:51.182629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4781e41-42c0-4396-9f37-be2bc9579268 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:51.194452 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1336/2658] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:30:51.200072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4828da-88c2-4fa7-915b-38cd4924e9cd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:51.209784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4828da-88c2-4fa7-915b-38cd4924e9cd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:51.214997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61adf7-add5-40be-9515-ac83ea2bf376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:51.215581 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61adf7-add5-40be-9515-ac83ea2bf376 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.215856 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61adf7-add5-40be-9515-ac83ea2bf376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.216120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61adf7-add5-40be-9515-ac83ea2bf376 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.216746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61adf7-add5-40be-9515-ac83ea2bf376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:51.218641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61adf7-add5-40be-9515-ac83ea2bf376 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:51.248436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1323/2659] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:51 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:51.252689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4781e41-42c0-4396-9f37-be2bc9579268 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:51.263734 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4781e41-42c0-4396-9f37-be2bc9579268 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:51.321022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71a77d-4b2a-471e-b2db-7910c862799d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:51.321634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71a77d-4b2a-471e-b2db-7910c862799d admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.321938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71a77d-4b2a-471e-b2db-7910c862799d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.322216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71a77d-4b2a-471e-b2db-7910c862799d admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.322856 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71a77d-4b2a-471e-b2db-7910c862799d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:51.325309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71a77d-4b2a-471e-b2db-7910c862799d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:51.347190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1337/2660] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:51 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:51.351342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61adf7-add5-40be-9515-ac83ea2bf376 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:51.361701 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61adf7-add5-40be-9515-ac83ea2bf376 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:51.366759 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7072472-b730-4f4e-b462-68018bc857b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:51.367458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7072472-b730-4f4e-b462-68018bc857b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.367833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7072472-b730-4f4e-b462-68018bc857b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.368213 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7072472-b730-4f4e-b462-68018bc857b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.369075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7072472-b730-4f4e-b462-68018bc857b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:51.371331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7072472-b730-4f4e-b462-68018bc857b5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:51.382110 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1324/2661] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:51.387483 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71a77d-4b2a-471e-b2db-7910c862799d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:51.397919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71a77d-4b2a-471e-b2db-7910c862799d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:51.404883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee281a2-259e-4a35-a49e-6fb6bb19b6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:51.405513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee281a2-259e-4a35-a49e-6fb6bb19b6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.405796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee281a2-259e-4a35-a49e-6fb6bb19b6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.406099 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee281a2-259e-4a35-a49e-6fb6bb19b6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.406757 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee281a2-259e-4a35-a49e-6fb6bb19b6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:51.408383 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee281a2-259e-4a35-a49e-6fb6bb19b6d6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:51.419603 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1338/2662] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:51.424112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7072472-b730-4f4e-b462-68018bc857b5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:51.434199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7072472-b730-4f4e-b462-68018bc857b5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:51.438850 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b8bb2-1e21-4e7d-ab9e-3800d08e9fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:51.439508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b8bb2-1e21-4e7d-ab9e-3800d08e9fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.439787 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b8bb2-1e21-4e7d-ab9e-3800d08e9fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.440062 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b8bb2-1e21-4e7d-ab9e-3800d08e9fb0 admin admin] PATH_INFO: `/v3/projects/7d7ea5a632494ebca76f2e2fcdc34e84/users/d48c98a3b8964ca293758859bf73ede8/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.455925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b8bb2-1e21-4e7d-ab9e-3800d08e9fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7ea5a632494ebca76f2e2fcdc34e84, user_id=d48c98a3b8964ca293758859bf73ede8, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:51.458616 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b8bb2-1e21-4e7d-ab9e-3800d08e9fb0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:51.473343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1325/2663] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:51 2022] PUT /identity/v3/projects/7d7ea5a632494ebca76f2e2fcdc34e84/users/d48c98a3b8964ca293758859bf73ede8/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:51.484946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777a7a3-d171-4feb-8189-93250600ad92 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.485489 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777a7a3-d171-4feb-8189-93250600ad92 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.485769 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777a7a3-d171-4feb-8189-93250600ad92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.516940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-a777a7a3-d171-4feb-8189-93250600ad92 None None] MFA Rules not processed for user `d48c98a3b8964ca293758859bf73ede8`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:51.518192 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a777a7a3-d171-4feb-8189-93250600ad92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:51.594494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1339/2664] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:51.599518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b8bb2-1e21-4e7d-ab9e-3800d08e9fb0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:51.670552 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b8bb2-1e21-4e7d-ab9e-3800d08e9fb0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:51.675380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff27c7e-38f4-40ee-be49-876ac77464b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:51.676000 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff27c7e-38f4-40ee-be49-876ac77464b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.676278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff27c7e-38f4-40ee-be49-876ac77464b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.676549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff27c7e-38f4-40ee-be49-876ac77464b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.677191 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff27c7e-38f4-40ee-be49-876ac77464b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:51.679130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff27c7e-38f4-40ee-be49-876ac77464b6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:51.709171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1326/2665] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:51 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:51.715368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777a7a3-d171-4feb-8189-93250600ad92 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:51.726941 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777a7a3-d171-4feb-8189-93250600ad92 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:51.784075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:51.793480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181adae-dfde-40c4-9b15-b350c1cd0733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:51.794627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181adae-dfde-40c4-9b15-b350c1cd0733 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.795129 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181adae-dfde-40c4-9b15-b350c1cd0733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.795830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181adae-dfde-40c4-9b15-b350c1cd0733 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.796908 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181adae-dfde-40c4-9b15-b350c1cd0733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:51.800404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181adae-dfde-40c4-9b15-b350c1cd0733 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:51.834418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1340/2666] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:51 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:30:51.838653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff27c7e-38f4-40ee-be49-876ac77464b6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:51.851303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff27c7e-38f4-40ee-be49-876ac77464b6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:51.857169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c372adb-839e-4bef-804b-680078669efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:51.857989 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c372adb-839e-4bef-804b-680078669efc admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.858305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c372adb-839e-4bef-804b-680078669efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.858574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c372adb-839e-4bef-804b-680078669efc admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.859222 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c372adb-839e-4bef-804b-680078669efc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:51.860967 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c372adb-839e-4bef-804b-680078669efc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:51.872905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1327/2667] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:51.878395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181adae-dfde-40c4-9b15-b350c1cd0733 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:51.890623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181adae-dfde-40c4-9b15-b350c1cd0733 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:51.895584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39faab3-2a8c-4112-a94a-cf63559aef45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:51.896244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39faab3-2a8c-4112-a94a-cf63559aef45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:51.896524 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39faab3-2a8c-4112-a94a-cf63559aef45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:51.896793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39faab3-2a8c-4112-a94a-cf63559aef45 admin admin] PATH_INFO: `/v3/projects/98929b9c9dd3401d9eb688429284d883/users/927fd0e523cb49eca4520f32732b7d33/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:51.913706 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39faab3-2a8c-4112-a94a-cf63559aef45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98929b9c9dd3401d9eb688429284d883, user_id=927fd0e523cb49eca4520f32732b7d33, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:51.916441 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39faab3-2a8c-4112-a94a-cf63559aef45 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:51.932170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1341/2668] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:51 2022] PUT /identity/v3/projects/98929b9c9dd3401d9eb688429284d883/users/927fd0e523cb49eca4520f32732b7d33/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:51.936600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c372adb-839e-4bef-804b-680078669efc admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:51.946997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c372adb-839e-4bef-804b-680078669efc admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:52.028160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cc027-f8a3-44db-86c4-0aa1914860f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:52.029188 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cc027-f8a3-44db-86c4-0aa1914860f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:52.029663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cc027-f8a3-44db-86c4-0aa1914860f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:52.030095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cc027-f8a3-44db-86c4-0aa1914860f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:52.030928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cc027-f8a3-44db-86c4-0aa1914860f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:52.033486 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cc027-f8a3-44db-86c4-0aa1914860f4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:52.045256 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1328/2669] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:52.050308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39faab3-2a8c-4112-a94a-cf63559aef45 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:52.063308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39faab3-2a8c-4112-a94a-cf63559aef45 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:52.068815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a23b6d-4214-445a-97dd-c2f0031befd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:52.069423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a23b6d-4214-445a-97dd-c2f0031befd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:52.069770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a23b6d-4214-445a-97dd-c2f0031befd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:52.070033 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a23b6d-4214-445a-97dd-c2f0031befd3 admin admin] PATH_INFO: `/v3/projects/98929b9c9dd3401d9eb688429284d883/users/927fd0e523cb49eca4520f32732b7d33/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:52.072268 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a23b6d-4214-445a-97dd-c2f0031befd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98929b9c9dd3401d9eb688429284d883, user_id=927fd0e523cb49eca4520f32732b7d33, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:52.076995 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a23b6d-4214-445a-97dd-c2f0031befd3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:52.090396 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1342/2670] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:52 2022] PUT /identity/v3/projects/98929b9c9dd3401d9eb688429284d883/users/927fd0e523cb49eca4520f32732b7d33/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:30:52.101194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0300be5-c1b5-4e19-98e2-a6f7440f0165 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:52.101494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0300be5-c1b5-4e19-98e2-a6f7440f0165 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:52.101757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0300be5-c1b5-4e19-98e2-a6f7440f0165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:52.117621 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:52.150794 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d0300be5-c1b5-4e19-98e2-a6f7440f0165 None None] MFA Rules not processed for user `927fd0e523cb49eca4520f32732b7d33`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:52.154283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d0300be5-c1b5-4e19-98e2-a6f7440f0165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:52.250412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1329/2671] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:52.263008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a23b6d-4214-445a-97dd-c2f0031befd3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:52.266639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-60a23b6d-4214-445a-97dd-c2f0031befd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:52.340021 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a23b6d-4214-445a-97dd-c2f0031befd3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:52.343782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ab262-8555-4ad7-a067-d58aaf5c6ef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:52.344462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ab262-8555-4ad7-a067-d58aaf5c6ef9 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:52.344793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ab262-8555-4ad7-a067-d58aaf5c6ef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:52.345068 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ab262-8555-4ad7-a067-d58aaf5c6ef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:52.353949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ab262-8555-4ad7-a067-d58aaf5c6ef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:52.355437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ab262-8555-4ad7-a067-d58aaf5c6ef9 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:52.365161 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1343/2672] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:30:52.847265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0300be5-c1b5-4e19-98e2-a6f7440f0165 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:52.933943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0300be5-c1b5-4e19-98e2-a6f7440f0165 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:52.940772 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:52.941562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:52.941876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:52.942241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] PATH_INFO: `/v3/users/d48c98a3b8964ca293758859bf73ede8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:52.943614 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d48c98a3b8964ca293758859bf73ede8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:52.945599 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:52.999997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd48c98a3b8964ca293758859bf73ede8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.008719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd48c98a3b8964ca293758859bf73ede8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.022963 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd48c98a3b8964ca293758859bf73ede8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.023678 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd48c98a3b8964ca293758859bf73ede8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.031780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd48c98a3b8964ca293758859bf73ede8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.037926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1330/2673] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:52 2022] DELETE /identity/v3/users/d48c98a3b8964ca293758859bf73ede8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:53.043027 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac5db8-6617-448a-b377-dc90ea46ea3c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:53.043371 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac5db8-6617-448a-b377-dc90ea46ea3c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:53.043807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac5db8-6617-448a-b377-dc90ea46ea3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:53.063945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-34ac5db8-6617-448a-b377-dc90ea46ea3c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:53.065169 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-34ac5db8-6617-448a-b377-dc90ea46ea3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:53.125445 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1344/2674] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:53.135580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:53.137601 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:53.153524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:53.201197 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9dc8d-f24b-4c52-bea2-23333b885e29 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:53.204714 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021b445-4f3c-42ca-bd47-a02c5c3441c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:53.205315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021b445-4f3c-42ca-bd47-a02c5c3441c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:53.205596 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021b445-4f3c-42ca-bd47-a02c5c3441c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:53.205868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021b445-4f3c-42ca-bd47-a02c5c3441c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:53.214994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021b445-4f3c-42ca-bd47-a02c5c3441c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:53.216417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021b445-4f3c-42ca-bd47-a02c5c3441c3 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:53.226086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1331/2675] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:53.501999 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac5db8-6617-448a-b377-dc90ea46ea3c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:53.503498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-34ac5db8-6617-448a-b377-dc90ea46ea3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:53.545135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac5db8-6617-448a-b377-dc90ea46ea3c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:53.555169 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:53.556087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:53.556772 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:53.557273 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] PATH_INFO: `/v3/projects/7d7ea5a632494ebca76f2e2fcdc34e84` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:53.559195 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7ea5a632494ebca76f2e2fcdc34e84)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:53.561671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:53.623406 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:53.626248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7ea5a632494ebca76f2e2fcdc34e84', 'request_id': 'req-c74a0307-3042-4327-9917-beeefcbee1f1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.640988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7ea5a632494ebca76f2e2fcdc34e84', 'request_id': 'req-c74a0307-3042-4327-9917-beeefcbee1f1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.649482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7ea5a632494ebca76f2e2fcdc34e84', 'request_id': 'req-c74a0307-3042-4327-9917-beeefcbee1f1'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.666164 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] The token cache is being invalidate because project 7d7ea5a632494ebca76f2e2fcdc34e84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:53.666679 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.667587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.669927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1345/2676] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:53 2022] DELETE /identity/v3/projects/7d7ea5a632494ebca76f2e2fcdc34e84 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:53.676568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021b445-4f3c-42ca-bd47-a02c5c3441c3 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:53.691131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021b445-4f3c-42ca-bd47-a02c5c3441c3 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:53.751707 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70142bc4-98f9-40df-9837-fdf257b19609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:53.752243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70142bc4-98f9-40df-9837-fdf257b19609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:53.752520 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70142bc4-98f9-40df-9837-fdf257b19609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:53.752790 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70142bc4-98f9-40df-9837-fdf257b19609 admin admin] PATH_INFO: `/v3/users/927fd0e523cb49eca4520f32732b7d33` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:53.753926 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70142bc4-98f9-40df-9837-fdf257b19609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=927fd0e523cb49eca4520f32732b7d33)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:53.755880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70142bc4-98f9-40df-9837-fdf257b19609 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:53.807183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70142bc4-98f9-40df-9837-fdf257b19609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '927fd0e523cb49eca4520f32732b7d33'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.816190 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70142bc4-98f9-40df-9837-fdf257b19609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '927fd0e523cb49eca4520f32732b7d33'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.828977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70142bc4-98f9-40df-9837-fdf257b19609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '927fd0e523cb49eca4520f32732b7d33'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.829615 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70142bc4-98f9-40df-9837-fdf257b19609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '927fd0e523cb49eca4520f32732b7d33'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.836400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70142bc4-98f9-40df-9837-fdf257b19609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '927fd0e523cb49eca4520f32732b7d33'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:53.842181 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1332/2677] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:53 2022] DELETE /identity/v3/users/927fd0e523cb49eca4520f32732b7d33 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:54.121014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:54.122596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:54.135963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74a0307-3042-4327-9917-beeefcbee1f1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:54.198540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d934eb8-285f-4380-8892-26d754a2e8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:54.199324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d934eb8-285f-4380-8892-26d754a2e8e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:54.199732 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d934eb8-285f-4380-8892-26d754a2e8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:54.200013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d934eb8-285f-4380-8892-26d754a2e8e4 admin admin] PATH_INFO: `/v3/projects/98929b9c9dd3401d9eb688429284d883` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:54.201153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d934eb8-285f-4380-8892-26d754a2e8e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98929b9c9dd3401d9eb688429284d883)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:54.203540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d934eb8-285f-4380-8892-26d754a2e8e4 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:54.255365 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5d934eb8-285f-4380-8892-26d754a2e8e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98929b9c9dd3401d9eb688429284d883', 'request_id': 'req-5d934eb8-285f-4380-8892-26d754a2e8e4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:54.264040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5d934eb8-285f-4380-8892-26d754a2e8e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98929b9c9dd3401d9eb688429284d883', 'request_id': 'req-5d934eb8-285f-4380-8892-26d754a2e8e4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:54.272180 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5d934eb8-285f-4380-8892-26d754a2e8e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98929b9c9dd3401d9eb688429284d883', 'request_id': 'req-5d934eb8-285f-4380-8892-26d754a2e8e4'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:54.287186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5d934eb8-285f-4380-8892-26d754a2e8e4 admin admin] The token cache is being invalidate because project 98929b9c9dd3401d9eb688429284d883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:54.287476 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5d934eb8-285f-4380-8892-26d754a2e8e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:54.288133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-5d934eb8-285f-4380-8892-26d754a2e8e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:54.289622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1346/2678] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:54 2022] DELETE /identity/v3/projects/98929b9c9dd3401d9eb688429284d883 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:54.294806 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8ac5b-8e0d-40fc-a69e-7e8f0c9462a3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:54.295333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8ac5b-8e0d-40fc-a69e-7e8f0c9462a3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:54.295333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8ac5b-8e0d-40fc-a69e-7e8f0c9462a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:54.315880 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-31b8ac5b-8e0d-40fc-a69e-7e8f0c9462a3 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:54.317234 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-31b8ac5b-8e0d-40fc-a69e-7e8f0c9462a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:54.349602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:54.381565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1333/2679] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:54.390436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e7f65-4254-40ef-8919-c85849a069f1 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:54.390902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e7f65-4254-40ef-8919-c85849a069f1 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:54.391200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e7f65-4254-40ef-8919-c85849a069f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:54.413097 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0f1e7f65-4254-40ef-8919-c85849a069f1 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:54.414341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0f1e7f65-4254-40ef-8919-c85849a069f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:54.422795 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1347/2680] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:54.426676 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8ac5b-8e0d-40fc-a69e-7e8f0c9462a3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:54.436292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8ac5b-8e0d-40fc-a69e-7e8f0c9462a3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:54.440798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a4f22-657d-400f-a008-a050e0f8436b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:54.441383 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a4f22-657d-400f-a008-a050e0f8436b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:54.441657 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a4f22-657d-400f-a008-a050e0f8436b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:54.441931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a4f22-657d-400f-a008-a050e0f8436b admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:54.442598 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a4f22-657d-400f-a008-a050e0f8436b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:54.442898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a4f22-657d-400f-a008-a050e0f8436b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:54.444464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a4f22-657d-400f-a008-a050e0f8436b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:54.456595 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1334/2681] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:30:54.460992 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e7f65-4254-40ef-8919-c85849a069f1 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:54.471070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e7f65-4254-40ef-8919-c85849a069f1 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:54.475708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e40c5-44f3-454e-9658-9d6cd05555f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:54.476327 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e40c5-44f3-454e-9658-9d6cd05555f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:54.476609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e40c5-44f3-454e-9658-9d6cd05555f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:54.476883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e40c5-44f3-454e-9658-9d6cd05555f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:54.477512 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e40c5-44f3-454e-9658-9d6cd05555f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:54.479523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e40c5-44f3-454e-9658-9d6cd05555f5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:54.507220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1348/2682] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:54 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:54.511975 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a4f22-657d-400f-a008-a050e0f8436b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:54.522089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a4f22-657d-400f-a008-a050e0f8436b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:54.578397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63c18e-922c-462f-b265-6cd03de822db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:54.578993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63c18e-922c-462f-b265-6cd03de822db admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:54.579296 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63c18e-922c-462f-b265-6cd03de822db admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:54.579568 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63c18e-922c-462f-b265-6cd03de822db admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:54.580185 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63c18e-922c-462f-b265-6cd03de822db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:54.582095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63c18e-922c-462f-b265-6cd03de822db admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:54.602323 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1335/2683] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:54 2022] POST /identity/v3/users => generated 281 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:54.606481 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e40c5-44f3-454e-9658-9d6cd05555f5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:54.616730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e40c5-44f3-454e-9658-9d6cd05555f5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:54.621834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714773b3-3359-484c-a174-74ede33a0bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:54.622523 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714773b3-3359-484c-a174-74ede33a0bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:54.622802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714773b3-3359-484c-a174-74ede33a0bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:54.623070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714773b3-3359-484c-a174-74ede33a0bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:54.623704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714773b3-3359-484c-a174-74ede33a0bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:54.625265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714773b3-3359-484c-a174-74ede33a0bf7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:54.635049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1349/2684] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:30:54.639492 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63c18e-922c-462f-b265-6cd03de822db admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:54.650095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63c18e-922c-462f-b265-6cd03de822db admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:54.655458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddb836-44e4-4b98-88e1-54dcdc6c2537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:54.656074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddb836-44e4-4b98-88e1-54dcdc6c2537 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:54.656358 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddb836-44e4-4b98-88e1-54dcdc6c2537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:54.656634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddb836-44e4-4b98-88e1-54dcdc6c2537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:54.657282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bddb836-44e4-4b98-88e1-54dcdc6c2537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:54.658902 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bddb836-44e4-4b98-88e1-54dcdc6c2537 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:54.671302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1336/2685] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:54.675561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714773b3-3359-484c-a174-74ede33a0bf7 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:54.690884 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714773b3-3359-484c-a174-74ede33a0bf7 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:54.696143 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc38aa-5225-4ee0-b2e6-eeae9acff29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:54.696932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc38aa-5225-4ee0-b2e6-eeae9acff29c admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:54.697215 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc38aa-5225-4ee0-b2e6-eeae9acff29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:54.697493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc38aa-5225-4ee0-b2e6-eeae9acff29c admin admin] PATH_INFO: `/v3/projects/413306d683694cc09628127d660c85c8/users/f21a869db78046cd9e8f253ef7e673d3/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:54.713383 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc38aa-5225-4ee0-b2e6-eeae9acff29c admin admin] RBAC: Authorizing `identity:create_grant(project_id=413306d683694cc09628127d660c85c8, user_id=f21a869db78046cd9e8f253ef7e673d3, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:54.716071 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dc38aa-5225-4ee0-b2e6-eeae9acff29c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:54.732501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1350/2686] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:54 2022] PUT /identity/v3/projects/413306d683694cc09628127d660c85c8/users/f21a869db78046cd9e8f253ef7e673d3/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:54.750344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a96b1-4360-42fb-b10e-ba49c186fdd0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:54.750344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a96b1-4360-42fb-b10e-ba49c186fdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:54.750344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a96b1-4360-42fb-b10e-ba49c186fdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:54.762503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc38aa-5225-4ee0-b2e6-eeae9acff29c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:54.764213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a9dc38aa-5225-4ee0-b2e6-eeae9acff29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:54.769679 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:54.774472 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7f0a96b1-4360-42fb-b10e-ba49c186fdd0 None None] MFA Rules not processed for user `f21a869db78046cd9e8f253ef7e673d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:54.775852 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7f0a96b1-4360-42fb-b10e-ba49c186fdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:54.828694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dc38aa-5225-4ee0-b2e6-eeae9acff29c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:54.832375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8877c-b122-47f4-8263-3820d1a5c06a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:54.832991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8877c-b122-47f4-8263-3820d1a5c06a service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:54.833391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8877c-b122-47f4-8263-3820d1a5c06a service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:54.833570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8877c-b122-47f4-8263-3820d1a5c06a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:54.835416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-10d8877c-b122-47f4-8263-3820d1a5c06a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:54.851747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1337/2687] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:30:54.888238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a96b1-4360-42fb-b10e-ba49c186fdd0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:54.889685 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7f0a96b1-4360-42fb-b10e-ba49c186fdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:54.902529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a96b1-4360-42fb-b10e-ba49c186fdd0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:54.904914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8877c-b122-47f4-8263-3820d1a5c06a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:54.905561 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8877c-b122-47f4-8263-3820d1a5c06a service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:54.906535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7c3ca-aed9-4ee8-8e2d-3bea58ba0265 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:54.907146 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7c3ca-aed9-4ee8-8e2d-3bea58ba0265 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:54.907455 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7c3ca-aed9-4ee8-8e2d-3bea58ba0265 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:54.907731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7c3ca-aed9-4ee8-8e2d-3bea58ba0265 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:54.915696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1351/2688] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:30:54.918961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7c3ca-aed9-4ee8-8e2d-3bea58ba0265 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:54.920408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7c3ca-aed9-4ee8-8e2d-3bea58ba0265 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:54.929934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1338/2689] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:30:54.945210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8877c-b122-47f4-8263-3820d1a5c06a service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:55.016762 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8877c-b122-47f4-8263-3820d1a5c06a service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:55.021261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:55.021896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:55.022213 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:55.022478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] PATH_INFO: `/v3/users/f21a869db78046cd9e8f253ef7e673d3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:55.023601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f21a869db78046cd9e8f253ef7e673d3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:55.025513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:55.078140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f21a869db78046cd9e8f253ef7e673d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:55.086290 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f21a869db78046cd9e8f253ef7e673d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:55.087897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f21a869db78046cd9e8f253ef7e673d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:55.089925 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:55.098559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f21a869db78046cd9e8f253ef7e673d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:55.111487 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f21a869db78046cd9e8f253ef7e673d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:55.117628 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1352/2690] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:54 2022] DELETE /identity/v3/users/f21a869db78046cd9e8f253ef7e673d3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:55.122692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dfb5f-2a5e-4aaa-b12a-4877d4248f14 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:55.123079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dfb5f-2a5e-4aaa-b12a-4877d4248f14 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:55.123397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dfb5f-2a5e-4aaa-b12a-4877d4248f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:55.145324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-aa7dfb5f-2a5e-4aaa-b12a-4877d4248f14 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:55.146577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-aa7dfb5f-2a5e-4aaa-b12a-4877d4248f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:55.168495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:55.227958 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1339/2691] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:55.234961 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:55.236388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:55.296475 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eaa1d9-f383-4824-aa65-cdd4a240f5d8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:55.300088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e479b-9161-4b37-948c-4527b48bc1ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:55.300698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e479b-9161-4b37-948c-4527b48bc1ce service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:55.300975 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e479b-9161-4b37-948c-4527b48bc1ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:55.301246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e479b-9161-4b37-948c-4527b48bc1ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:55.310358 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e479b-9161-4b37-948c-4527b48bc1ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:55.311791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e479b-9161-4b37-948c-4527b48bc1ce service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:55.321233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1353/2692] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:55.602489 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dfb5f-2a5e-4aaa-b12a-4877d4248f14 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:55.603970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-aa7dfb5f-2a5e-4aaa-b12a-4877d4248f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:55.618498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dfb5f-2a5e-4aaa-b12a-4877d4248f14 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:55.623334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a259934-c998-4664-9af9-4482c1f6f237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:55.624120 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a259934-c998-4664-9af9-4482c1f6f237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:55.624396 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a259934-c998-4664-9af9-4482c1f6f237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:55.624659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a259934-c998-4664-9af9-4482c1f6f237 admin admin] PATH_INFO: `/v3/projects/413306d683694cc09628127d660c85c8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:55.625779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a259934-c998-4664-9af9-4482c1f6f237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=413306d683694cc09628127d660c85c8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:55.627756 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a259934-c998-4664-9af9-4482c1f6f237 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:55.678336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4a259934-c998-4664-9af9-4482c1f6f237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '413306d683694cc09628127d660c85c8', 'request_id': 'req-4a259934-c998-4664-9af9-4482c1f6f237'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:55.686078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4a259934-c998-4664-9af9-4482c1f6f237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '413306d683694cc09628127d660c85c8', 'request_id': 'req-4a259934-c998-4664-9af9-4482c1f6f237'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:55.692747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4a259934-c998-4664-9af9-4482c1f6f237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '413306d683694cc09628127d660c85c8', 'request_id': 'req-4a259934-c998-4664-9af9-4482c1f6f237'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:55.705334 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4a259934-c998-4664-9af9-4482c1f6f237 admin admin] The token cache is being invalidate because project 413306d683694cc09628127d660c85c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:30:55.705620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4a259934-c998-4664-9af9-4482c1f6f237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:55.706289 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4a259934-c998-4664-9af9-4482c1f6f237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:30:55.707747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1340/2693] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:30:55 2022] DELETE /identity/v3/projects/413306d683694cc09628127d660c85c8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:55.712749 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97282e-7f30-4018-9324-066ae69a931c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:55.713052 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97282e-7f30-4018-9324-066ae69a931c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:55.713344 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b97282e-7f30-4018-9324-066ae69a931c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:55.733710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9b97282e-7f30-4018-9324-066ae69a931c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:55.734960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9b97282e-7f30-4018-9324-066ae69a931c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:55.795650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1354/2694] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:55.804437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07507fee-e081-4f2c-acbf-de554e6f4ee4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:55.804728 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07507fee-e081-4f2c-acbf-de554e6f4ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:55.805071 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07507fee-e081-4f2c-acbf-de554e6f4ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:55.826100 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-07507fee-e081-4f2c-acbf-de554e6f4ee4 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:55.827360 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-07507fee-e081-4f2c-acbf-de554e6f4ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:55.835703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1341/2695] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:55.840703 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97282e-7f30-4018-9324-066ae69a931c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:55.850526 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b97282e-7f30-4018-9324-066ae69a931c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:55.855233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e30654-418c-41f6-b833-b2d1001716cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:55.855826 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e30654-418c-41f6-b833-b2d1001716cc admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:55.856106 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e30654-418c-41f6-b833-b2d1001716cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:55.856912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e30654-418c-41f6-b833-b2d1001716cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:55.857050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e30654-418c-41f6-b833-b2d1001716cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:30:55.857351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e30654-418c-41f6-b833-b2d1001716cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:55.859283 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e30654-418c-41f6-b833-b2d1001716cc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:55.870877 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1355/2696] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:30:55.875499 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07507fee-e081-4f2c-acbf-de554e6f4ee4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:55.885311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07507fee-e081-4f2c-acbf-de554e6f4ee4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:55.893023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3b29d-e6c1-4f51-b3f8-a8f1d8aa71e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:55.893937 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3b29d-e6c1-4f51-b3f8-a8f1d8aa71e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:55.894261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3b29d-e6c1-4f51-b3f8-a8f1d8aa71e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:55.894537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3b29d-e6c1-4f51-b3f8-a8f1d8aa71e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:55.895216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3b29d-e6c1-4f51-b3f8-a8f1d8aa71e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:55.897125 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3b29d-e6c1-4f51-b3f8-a8f1d8aa71e8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:55.926986 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1342/2697] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:30:55 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:55.931143 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e30654-418c-41f6-b833-b2d1001716cc admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:55.941736 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e30654-418c-41f6-b833-b2d1001716cc admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:55.998226 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234661b-9c89-42bc-a040-59875be2820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:55.998837 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234661b-9c89-42bc-a040-59875be2820a admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:55.999118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234661b-9c89-42bc-a040-59875be2820a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:55.999391 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234661b-9c89-42bc-a040-59875be2820a admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:56.000018 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234661b-9c89-42bc-a040-59875be2820a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:56.001914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234661b-9c89-42bc-a040-59875be2820a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:56.010879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:56.024973 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1356/2698] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:30:55 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:30:56.029962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3b29d-e6c1-4f51-b3f8-a8f1d8aa71e8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:56.040317 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3b29d-e6c1-4f51-b3f8-a8f1d8aa71e8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:56.045518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca9dc0-3672-4a84-9d97-d6ef544521c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:56.046148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca9dc0-3672-4a84-9d97-d6ef544521c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:56.046426 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca9dc0-3672-4a84-9d97-d6ef544521c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:56.046698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca9dc0-3672-4a84-9d97-d6ef544521c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:56.047335 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca9dc0-3672-4a84-9d97-d6ef544521c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:56.048941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca9dc0-3672-4a84-9d97-d6ef544521c2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:56.059018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1343/2699] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:56.062881 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234661b-9c89-42bc-a040-59875be2820a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:56.072650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234661b-9c89-42bc-a040-59875be2820a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:56.077472 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea9c93-a754-4181-9e35-d0368d5a31d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:56.078082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea9c93-a754-4181-9e35-d0368d5a31d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:56.078362 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea9c93-a754-4181-9e35-d0368d5a31d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:56.078630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea9c93-a754-4181-9e35-d0368d5a31d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:56.079279 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea9c93-a754-4181-9e35-d0368d5a31d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:56.080827 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea9c93-a754-4181-9e35-d0368d5a31d6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:56.091876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1357/2700] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:30:56.095936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca9dc0-3672-4a84-9d97-d6ef544521c2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:56.106293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca9dc0-3672-4a84-9d97-d6ef544521c2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:56.110802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aad957-3349-4462-9ff2-818a66d3493a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:56.111437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aad957-3349-4462-9ff2-818a66d3493a admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:56.111725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aad957-3349-4462-9ff2-818a66d3493a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:56.111979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aad957-3349-4462-9ff2-818a66d3493a admin admin] PATH_INFO: `/v3/projects/80f7b40b069047cebea956cf792c0bb9/users/e8ecf5615659422abf80f05686c39e97/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:56.126238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aad957-3349-4462-9ff2-818a66d3493a admin admin] RBAC: Authorizing `identity:create_grant(project_id=80f7b40b069047cebea956cf792c0bb9, user_id=e8ecf5615659422abf80f05686c39e97, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:56.128845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aad957-3349-4462-9ff2-818a66d3493a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:56.141985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1344/2701] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:30:56 2022] PUT /identity/v3/projects/80f7b40b069047cebea956cf792c0bb9/users/e8ecf5615659422abf80f05686c39e97/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:30:56.153098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9980b-97b2-411e-a3fd-2b4078780a14 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:56.153528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9980b-97b2-411e-a3fd-2b4078780a14 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:56.153836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9980b-97b2-411e-a3fd-2b4078780a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:56.182390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-2cd9980b-97b2-411e-a3fd-2b4078780a14 None None] MFA Rules not processed for user `e8ecf5615659422abf80f05686c39e97`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:30:56.183676 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2cd9980b-97b2-411e-a3fd-2b4078780a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:56.250400 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1358/2702] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:30:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:30:56.258608 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aad957-3349-4462-9ff2-818a66d3493a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:56.260481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d1aad957-3349-4462-9ff2-818a66d3493a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:56.309211 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:30:56.332344 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aad957-3349-4462-9ff2-818a66d3493a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:56.336177 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6768e-bd19-467f-bcff-3b4660f9f405 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2d76fb9cbdd40f9b2a748209cee5b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:56.336797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6768e-bd19-467f-bcff-3b4660f9f405 service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:56.337077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6768e-bd19-467f-bcff-3b4660f9f405 service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:56.337351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6768e-bd19-467f-bcff-3b4660f9f405 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:56.348654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6768e-bd19-467f-bcff-3b4660f9f405 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:30:56.349876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6768e-bd19-467f-bcff-3b4660f9f405 service glance] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:30:56.361173 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1345/2703] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:30:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:30:59.895644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9980b-97b2-411e-a3fd-2b4078780a14 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:30:59.897942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2cd9980b-97b2-411e-a3fd-2b4078780a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:30:59.991221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9980b-97b2-411e-a3fd-2b4078780a14 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:30:59.996198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:30:59.996848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:30:59.997126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:30:59.997403 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] PATH_INFO: `/v3/users/56f5ca121eeb4c29a2c85a02078c8b2c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:30:59.998599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] RBAC: Authorizing `identity:delete_user(user_id=56f5ca121eeb4c29a2c85a02078c8b2c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:00.000538 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:00.058697 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56f5ca121eeb4c29a2c85a02078c8b2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.069708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56f5ca121eeb4c29a2c85a02078c8b2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.070435 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56f5ca121eeb4c29a2c85a02078c8b2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.078832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56f5ca121eeb4c29a2c85a02078c8b2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.094979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:00.101443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56f5ca121eeb4c29a2c85a02078c8b2c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.110775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1359/2704] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:30:59 2022] DELETE /identity/v3/users/56f5ca121eeb4c29a2c85a02078c8b2c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:00.116459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe445ac-035f-4e6a-b88c-0a1a4034950a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:00.117055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe445ac-035f-4e6a-b88c-0a1a4034950a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:00.117484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe445ac-035f-4e6a-b88c-0a1a4034950a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:00.150176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-abe445ac-035f-4e6a-b88c-0a1a4034950a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:00.152793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-abe445ac-035f-4e6a-b88c-0a1a4034950a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:00.231238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1346/2705] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:00.243805 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:00.245209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:00.317203 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef81721-d3ef-4710-bb96-0b383f75e19a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:00.321023 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d68916-cb59-4c94-88ac-1c071ebf178d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:00.321652 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d68916-cb59-4c94-88ac-1c071ebf178d service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:00.321929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d68916-cb59-4c94-88ac-1c071ebf178d service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:00.322237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d68916-cb59-4c94-88ac-1c071ebf178d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:00.332662 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d68916-cb59-4c94-88ac-1c071ebf178d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:00.334150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d68916-cb59-4c94-88ac-1c071ebf178d service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:00.346791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1360/2706] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:31:00.659272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe445ac-035f-4e6a-b88c-0a1a4034950a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:00.661051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-abe445ac-035f-4e6a-b88c-0a1a4034950a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:00.673839 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe445ac-035f-4e6a-b88c-0a1a4034950a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:00.678458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:00.679119 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:00.679380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:00.679654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] PATH_INFO: `/v3/projects/b00bbf6ff5d24c98a69f4cc78fd82051` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:00.680995 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b00bbf6ff5d24c98a69f4cc78fd82051)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:00.683484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:00.738485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b00bbf6ff5d24c98a69f4cc78fd82051', 'request_id': 'req-510f286d-d31a-4c97-af17-e414baea84cc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.749285 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b00bbf6ff5d24c98a69f4cc78fd82051', 'request_id': 'req-510f286d-d31a-4c97-af17-e414baea84cc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.757709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b00bbf6ff5d24c98a69f4cc78fd82051', 'request_id': 'req-510f286d-d31a-4c97-af17-e414baea84cc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.770605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] The token cache is being invalidate because project b00bbf6ff5d24c98a69f4cc78fd82051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:31:00.770845 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.772634 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.772922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1347/2707] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:31:00 2022] DELETE /identity/v3/projects/b00bbf6ff5d24c98a69f4cc78fd82051 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:00.777544 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d68916-cb59-4c94-88ac-1c071ebf178d service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:00.787691 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d68916-cb59-4c94-88ac-1c071ebf178d service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:00.854599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a83180-0d9b-4f0a-a8d2-f31ef09b5c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:00.855276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83180-0d9b-4f0a-a8d2-f31ef09b5c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:00.855550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83180-0d9b-4f0a-a8d2-f31ef09b5c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:00.855818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a83180-0d9b-4f0a-a8d2-f31ef09b5c78 admin admin] PATH_INFO: `/v3/users/29399e1aa91549ac93d5cc0f4ef88a7d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:00.857009 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a83180-0d9b-4f0a-a8d2-f31ef09b5c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29399e1aa91549ac93d5cc0f4ef88a7d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:00.858982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a83180-0d9b-4f0a-a8d2-f31ef09b5c78 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:00.918988 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-89a83180-0d9b-4f0a-a8d2-f31ef09b5c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29399e1aa91549ac93d5cc0f4ef88a7d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.937997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-89a83180-0d9b-4f0a-a8d2-f31ef09b5c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29399e1aa91549ac93d5cc0f4ef88a7d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.938765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-89a83180-0d9b-4f0a-a8d2-f31ef09b5c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29399e1aa91549ac93d5cc0f4ef88a7d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.946911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-89a83180-0d9b-4f0a-a8d2-f31ef09b5c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29399e1aa91549ac93d5cc0f4ef88a7d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.961868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-89a83180-0d9b-4f0a-a8d2-f31ef09b5c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29399e1aa91549ac93d5cc0f4ef88a7d'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:00.970448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1361/2708] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:31:00 2022] DELETE /identity/v3/users/29399e1aa91549ac93d5cc0f4ef88a7d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:01.327717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:01.335993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:01.359022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f286d-d31a-4c97-af17-e414baea84cc admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:01.444526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:01.444526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:01.445001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:01.445165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862 admin admin] PATH_INFO: `/v3/projects/40169ef9f8e24f75a92fbc2e07f61b76` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:01.446516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40169ef9f8e24f75a92fbc2e07f61b76)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:01.449200 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:01.452214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:01.519037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40169ef9f8e24f75a92fbc2e07f61b76', 'request_id': 'req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:01.534240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40169ef9f8e24f75a92fbc2e07f61b76', 'request_id': 'req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:01.543915 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40169ef9f8e24f75a92fbc2e07f61b76', 'request_id': 'req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:01.561824 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862 admin admin] The token cache is being invalidate because project 40169ef9f8e24f75a92fbc2e07f61b76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:31:01.562155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:01.565271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e0fba216-7dc6-4fc3-b6d3-7dd819fc2862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:01.565271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1348/2709] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:31:01 2022] DELETE /identity/v3/projects/40169ef9f8e24f75a92fbc2e07f61b76 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:01.570843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a7a70-34ec-4cfe-9c2c-0859b3b2e827 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:01.571108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a7a70-34ec-4cfe-9c2c-0859b3b2e827 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:01.571261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42a7a70-34ec-4cfe-9c2c-0859b3b2e827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:01.595811 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c42a7a70-34ec-4cfe-9c2c-0859b3b2e827 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:01.597207 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c42a7a70-34ec-4cfe-9c2c-0859b3b2e827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:01.604659 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:01.681493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1362/2710] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:31:01.694345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e739-4e82-4da7-bec4-157a791ac53d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:01.694629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e739-4e82-4da7-bec4-157a791ac53d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:01.694974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e739-4e82-4da7-bec4-157a791ac53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:01.721765 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1ff9e739-4e82-4da7-bec4-157a791ac53d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:01.723036 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1ff9e739-4e82-4da7-bec4-157a791ac53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:01.733357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1349/2711] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:01.737747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a7a70-34ec-4cfe-9c2c-0859b3b2e827 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:01.748953 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42a7a70-34ec-4cfe-9c2c-0859b3b2e827 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:01.754775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74051c62-0eb3-40d5-b4f1-2cb4018439a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:01.755428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74051c62-0eb3-40d5-b4f1-2cb4018439a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:01.755711 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74051c62-0eb3-40d5-b4f1-2cb4018439a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:01.755989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74051c62-0eb3-40d5-b4f1-2cb4018439a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:01.756678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74051c62-0eb3-40d5-b4f1-2cb4018439a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:31:01.756987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74051c62-0eb3-40d5-b4f1-2cb4018439a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:01.758966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74051c62-0eb3-40d5-b4f1-2cb4018439a5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:01.773960 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1363/2712] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:31:01.779175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9e739-4e82-4da7-bec4-157a791ac53d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:01.789076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9e739-4e82-4da7-bec4-157a791ac53d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:01.794517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860461db-d78a-4987-8592-dbcfc9555226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:01.795174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860461db-d78a-4987-8592-dbcfc9555226 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:01.795458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860461db-d78a-4987-8592-dbcfc9555226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:01.795765 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860461db-d78a-4987-8592-dbcfc9555226 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:01.796387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860461db-d78a-4987-8592-dbcfc9555226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:01.798417 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860461db-d78a-4987-8592-dbcfc9555226 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:01.841231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1350/2713] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:31:01 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:31:01.846196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74051c62-0eb3-40d5-b4f1-2cb4018439a5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:01.865311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74051c62-0eb3-40d5-b4f1-2cb4018439a5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:01.956663 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b3a05-54bd-4ffb-877f-51564d4bbaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:01.957555 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b3a05-54bd-4ffb-877f-51564d4bbaa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:01.957917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b3a05-54bd-4ffb-877f-51564d4bbaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:01.958331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b3a05-54bd-4ffb-877f-51564d4bbaa6 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:01.959275 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b3a05-54bd-4ffb-877f-51564d4bbaa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:01.962742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b3a05-54bd-4ffb-877f-51564d4bbaa6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:01.992426 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1364/2714] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:31:01 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:31:02.001171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860461db-d78a-4987-8592-dbcfc9555226 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:02.020341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860461db-d78a-4987-8592-dbcfc9555226 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:02.029094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819c980-206a-4e2b-bf44-b28fc9994d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:02.029914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819c980-206a-4e2b-bf44-b28fc9994d7c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:02.030326 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819c980-206a-4e2b-bf44-b28fc9994d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:02.030694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819c980-206a-4e2b-bf44-b28fc9994d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:02.031504 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819c980-206a-4e2b-bf44-b28fc9994d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:02.033484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819c980-206a-4e2b-bf44-b28fc9994d7c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:02.046247 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1351/2715] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:31:02.050644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b3a05-54bd-4ffb-877f-51564d4bbaa6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:02.066573 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b3a05-54bd-4ffb-877f-51564d4bbaa6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:02.076602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01db23-8e3d-42e8-a7fa-3df4d3f41594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:02.077571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01db23-8e3d-42e8-a7fa-3df4d3f41594 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:02.078188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01db23-8e3d-42e8-a7fa-3df4d3f41594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:02.078642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01db23-8e3d-42e8-a7fa-3df4d3f41594 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:02.079892 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01db23-8e3d-42e8-a7fa-3df4d3f41594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:02.083583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d01db23-8e3d-42e8-a7fa-3df4d3f41594 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:02.102229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1365/2716] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:31:02.110078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819c980-206a-4e2b-bf44-b28fc9994d7c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:02.128953 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819c980-206a-4e2b-bf44-b28fc9994d7c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:02.137169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a517063-62df-444f-b42d-b91affaedb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:02.137911 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a517063-62df-444f-b42d-b91affaedb3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:02.138246 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a517063-62df-444f-b42d-b91affaedb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:02.138529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a517063-62df-444f-b42d-b91affaedb3f admin admin] PATH_INFO: `/v3/projects/927fcbfca61049018c42eb9cda6f8f48/users/0075be376b3e497ba1574d54043315f3/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:02.171458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a517063-62df-444f-b42d-b91affaedb3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=927fcbfca61049018c42eb9cda6f8f48, user_id=0075be376b3e497ba1574d54043315f3, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:02.178789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a517063-62df-444f-b42d-b91affaedb3f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:02.210816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1352/2717] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:31:02 2022] PUT /identity/v3/projects/927fcbfca61049018c42eb9cda6f8f48/users/0075be376b3e497ba1574d54043315f3/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:02.231228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd583d4-0eee-4f9d-ac32-0a65c85506b7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:02.231718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd583d4-0eee-4f9d-ac32-0a65c85506b7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:02.232478 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd583d4-0eee-4f9d-ac32-0a65c85506b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:02.321458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ddd583d4-0eee-4f9d-ac32-0a65c85506b7 None None] MFA Rules not processed for user `0075be376b3e497ba1574d54043315f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:02.324838 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ddd583d4-0eee-4f9d-ac32-0a65c85506b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:02.431576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1366/2718] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:02.437652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a517063-62df-444f-b42d-b91affaedb3f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:02.554848 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a517063-62df-444f-b42d-b91affaedb3f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:02.563731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708ab98-8275-4488-9265-371b0841eec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:02.564944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708ab98-8275-4488-9265-371b0841eec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:02.565529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708ab98-8275-4488-9265-371b0841eec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:02.565925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708ab98-8275-4488-9265-371b0841eec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:02.568672 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708ab98-8275-4488-9265-371b0841eec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:02.576395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a708ab98-8275-4488-9265-371b0841eec9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:02.651878 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1353/2719] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:31:02 2022] POST /identity/v3/projects => generated 375 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:31:02.658573 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd583d4-0eee-4f9d-ac32-0a65c85506b7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:02.685644 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd583d4-0eee-4f9d-ac32-0a65c85506b7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:02.707329 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:02.875930 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862d5c2-c547-4cfb-aea9-0e8e2c737de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:02.876179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862d5c2-c547-4cfb-aea9-0e8e2c737de3 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:02.876499 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862d5c2-c547-4cfb-aea9-0e8e2c737de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:02.876910 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862d5c2-c547-4cfb-aea9-0e8e2c737de3 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:02.877705 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862d5c2-c547-4cfb-aea9-0e8e2c737de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:02.883075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862d5c2-c547-4cfb-aea9-0e8e2c737de3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:02.952592 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1367/2720] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:31:02 2022] POST /identity/v3/users => generated 288 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:31:02.957943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708ab98-8275-4488-9265-371b0841eec9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:02.984668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a708ab98-8275-4488-9265-371b0841eec9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:02.993906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ded16-ebe1-406b-8850-d38e1e8ceb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:02.996047 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ded16-ebe1-406b-8850-d38e1e8ceb55 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:02.996654 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ded16-ebe1-406b-8850-d38e1e8ceb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:02.996942 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ded16-ebe1-406b-8850-d38e1e8ceb55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:02.997696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ded16-ebe1-406b-8850-d38e1e8ceb55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:03.000330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ded16-ebe1-406b-8850-d38e1e8ceb55 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:03.021021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1354/2721] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:31:03.025565 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862d5c2-c547-4cfb-aea9-0e8e2c737de3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:03.037947 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862d5c2-c547-4cfb-aea9-0e8e2c737de3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:03.044348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e2004-b97e-4828-85a7-d448910f49cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:03.044986 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e2004-b97e-4828-85a7-d448910f49cb admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:03.045459 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e2004-b97e-4828-85a7-d448910f49cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:03.045576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e2004-b97e-4828-85a7-d448910f49cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:03.046269 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e2004-b97e-4828-85a7-d448910f49cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:03.048126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e2004-b97e-4828-85a7-d448910f49cb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:03.062879 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1368/2722] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:31:03.068144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ded16-ebe1-406b-8850-d38e1e8ceb55 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:03.078774 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:03.086160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ded16-ebe1-406b-8850-d38e1e8ceb55 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:03.096207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf671f-3366-4d4e-bc93-ac9814270ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:03.096894 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf671f-3366-4d4e-bc93-ac9814270ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:03.097171 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf671f-3366-4d4e-bc93-ac9814270ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:03.097444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf671f-3366-4d4e-bc93-ac9814270ec5 admin admin] PATH_INFO: `/v3/projects/050233c3bd1e48f081ef450ba21f546d/users/6a47ddf6d17443ff8ca9b25c41107ea9/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:03.118564 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf671f-3366-4d4e-bc93-ac9814270ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=050233c3bd1e48f081ef450ba21f546d, user_id=6a47ddf6d17443ff8ca9b25c41107ea9, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:03.122252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf671f-3366-4d4e-bc93-ac9814270ec5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:03.144141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1355/2723] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:31:03 2022] PUT /identity/v3/projects/050233c3bd1e48f081ef450ba21f546d/users/6a47ddf6d17443ff8ca9b25c41107ea9/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:03.157295 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f80aa-2db9-49c4-9bc8-b4822a9a3e68 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:03.157620 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f80aa-2db9-49c4-9bc8-b4822a9a3e68 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:03.158079 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f80aa-2db9-49c4-9bc8-b4822a9a3e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:03.194658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-498f80aa-2db9-49c4-9bc8-b4822a9a3e68 None None] MFA Rules not processed for user `6a47ddf6d17443ff8ca9b25c41107ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:03.195809 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-498f80aa-2db9-49c4-9bc8-b4822a9a3e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:03.285080 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1369/2724] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:31:03.290414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf671f-3366-4d4e-bc93-ac9814270ec5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:03.402524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf671f-3366-4d4e-bc93-ac9814270ec5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:03.413159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c17f9-077d-45bc-a42d-3d2994c97898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:03.413797 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c17f9-077d-45bc-a42d-3d2994c97898 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:03.414463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c17f9-077d-45bc-a42d-3d2994c97898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:03.414463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c17f9-077d-45bc-a42d-3d2994c97898 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:03.415008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c17f9-077d-45bc-a42d-3d2994c97898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:03.417553 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c17f9-077d-45bc-a42d-3d2994c97898 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:03.464278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1356/2725] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:31:03 2022] POST /identity/v3/projects => generated 375 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:31:03.466234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f80aa-2db9-49c4-9bc8-b4822a9a3e68 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:03.483373 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f80aa-2db9-49c4-9bc8-b4822a9a3e68 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:03.580595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971dd61-cdff-419a-b393-4f28e4bbd45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:03.581381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971dd61-cdff-419a-b393-4f28e4bbd45c admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:03.581773 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971dd61-cdff-419a-b393-4f28e4bbd45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:03.582272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971dd61-cdff-419a-b393-4f28e4bbd45c admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:03.583065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971dd61-cdff-419a-b393-4f28e4bbd45c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:03.585257 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971dd61-cdff-419a-b393-4f28e4bbd45c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:03.619327 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1370/2726] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:31:03 2022] POST /identity/v3/users => generated 294 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:31:03.623470 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c17f9-077d-45bc-a42d-3d2994c97898 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:03.638719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c17f9-077d-45bc-a42d-3d2994c97898 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:03.647347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860fd82-8413-4628-a66e-0ad30ebd4db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:03.649400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860fd82-8413-4628-a66e-0ad30ebd4db1 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:03.650011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860fd82-8413-4628-a66e-0ad30ebd4db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:03.651172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860fd82-8413-4628-a66e-0ad30ebd4db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:03.661814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860fd82-8413-4628-a66e-0ad30ebd4db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:03.665456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860fd82-8413-4628-a66e-0ad30ebd4db1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:03.707431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1357/2727] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:31:03.713598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971dd61-cdff-419a-b393-4f28e4bbd45c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:03.722345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:03.727436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971dd61-cdff-419a-b393-4f28e4bbd45c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:03.733852 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1f226-b13f-493c-98af-caae20f47ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:03.734640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f226-b13f-493c-98af-caae20f47ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:03.735160 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f226-b13f-493c-98af-caae20f47ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:03.735350 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f226-b13f-493c-98af-caae20f47ff5 admin admin] PATH_INFO: `/v3/projects/1a0823386d2f4c74b15f421fd1608fa9/users/3244b3fb7f9045a8a137e0ca3ddb6ab2/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:03.758779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1f226-b13f-493c-98af-caae20f47ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0823386d2f4c74b15f421fd1608fa9, user_id=3244b3fb7f9045a8a137e0ca3ddb6ab2, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:03.763065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a1f226-b13f-493c-98af-caae20f47ff5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:03.788303 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1371/2728] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:31:03 2022] PUT /identity/v3/projects/1a0823386d2f4c74b15f421fd1608fa9/users/3244b3fb7f9045a8a137e0ca3ddb6ab2/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:03.788843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860fd82-8413-4628-a66e-0ad30ebd4db1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:03.808924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860fd82-8413-4628-a66e-0ad30ebd4db1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:03.935800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e944a-0a82-48fc-94c3-f06ab103dc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:03.936872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e944a-0a82-48fc-94c3-f06ab103dc14 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:03.938006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e944a-0a82-48fc-94c3-f06ab103dc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:03.938006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e944a-0a82-48fc-94c3-f06ab103dc14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:03.939110 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e944a-0a82-48fc-94c3-f06ab103dc14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:03.942252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e944a-0a82-48fc-94c3-f06ab103dc14 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:03.962351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1358/2729] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:03 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:31:03.965480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1f226-b13f-493c-98af-caae20f47ff5 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:03.984683 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1f226-b13f-493c-98af-caae20f47ff5 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:03.993977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad9069-d6f0-49a3-a9fd-a9f3fade5864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:03.995666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad9069-d6f0-49a3-a9fd-a9f3fade5864 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:03.996409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad9069-d6f0-49a3-a9fd-a9f3fade5864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:03.996902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad9069-d6f0-49a3-a9fd-a9f3fade5864 admin admin] PATH_INFO: `/v3/projects/1a0823386d2f4c74b15f421fd1608fa9/users/3244b3fb7f9045a8a137e0ca3ddb6ab2/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:04.000846 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad9069-d6f0-49a3-a9fd-a9f3fade5864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0823386d2f4c74b15f421fd1608fa9, user_id=3244b3fb7f9045a8a137e0ca3ddb6ab2, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:04.006435 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad9069-d6f0-49a3-a9fd-a9f3fade5864 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:04.034857 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1372/2730] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:31:03 2022] PUT /identity/v3/projects/1a0823386d2f4c74b15f421fd1608fa9/users/3244b3fb7f9045a8a137e0ca3ddb6ab2/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:04.046393 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16835187-b4f1-45b9-9b08-afc01dce1854 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:04.046881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16835187-b4f1-45b9-9b08-afc01dce1854 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:04.047255 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16835187-b4f1-45b9-9b08-afc01dce1854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:04.089680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-16835187-b4f1-45b9-9b08-afc01dce1854 None None] MFA Rules not processed for user `3244b3fb7f9045a8a137e0ca3ddb6ab2`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:04.098567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-16835187-b4f1-45b9-9b08-afc01dce1854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:04.169498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:04.247095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1359/2731] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:31:04.260745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad9069-d6f0-49a3-a9fd-a9f3fade5864 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:04.264618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-afad9069-d6f0-49a3-a9fd-a9f3fade5864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:04.356172 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad9069-d6f0-49a3-a9fd-a9f3fade5864 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:04.360137 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6505790-9d12-45d0-ac81-47b0a3225be2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:04.360772 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6505790-9d12-45d0-ac81-47b0a3225be2 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:04.361190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6505790-9d12-45d0-ac81-47b0a3225be2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:04.361368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6505790-9d12-45d0-ac81-47b0a3225be2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:04.456219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6505790-9d12-45d0-ac81-47b0a3225be2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:04.457742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6505790-9d12-45d0-ac81-47b0a3225be2 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:04.477106 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1373/2732] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:31:06.166570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16835187-b4f1-45b9-9b08-afc01dce1854 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:06.167994 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-16835187-b4f1-45b9-9b08-afc01dce1854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:06.180456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16835187-b4f1-45b9-9b08-afc01dce1854 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:06.184084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f383f-3157-4351-946c-5b137d1b2809 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:06.184688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f383f-3157-4351-946c-5b137d1b2809 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:06.184964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f383f-3157-4351-946c-5b137d1b2809 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:06.185237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f383f-3157-4351-946c-5b137d1b2809 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:06.252350 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f383f-3157-4351-946c-5b137d1b2809 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:06.253711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f383f-3157-4351-946c-5b137d1b2809 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:06.263933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1360/2733] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:31:06.892247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6505790-9d12-45d0-ac81-47b0a3225be2 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:06.902693 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6505790-9d12-45d0-ac81-47b0a3225be2 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:06.907135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145d702-961c-48f0-a14a-0ed72e988885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:06.907758 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145d702-961c-48f0-a14a-0ed72e988885 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:06.908037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145d702-961c-48f0-a14a-0ed72e988885 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:06.908306 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1145d702-961c-48f0-a14a-0ed72e988885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:06.921875 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145d702-961c-48f0-a14a-0ed72e988885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:06.923587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1145d702-961c-48f0-a14a-0ed72e988885 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:06.935917 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1374/2734] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:31:07.043788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f383f-3157-4351-946c-5b137d1b2809 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:07.045550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ca2f383f-3157-4351-946c-5b137d1b2809 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:07.135138 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f383f-3157-4351-946c-5b137d1b2809 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:07.140721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f448db2-bdd8-4183-a8a5-4043724eb98b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:07.141451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f448db2-bdd8-4183-a8a5-4043724eb98b service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:07.141750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f448db2-bdd8-4183-a8a5-4043724eb98b service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:07.142133 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f448db2-bdd8-4183-a8a5-4043724eb98b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:07.144245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0f448db2-bdd8-4183-a8a5-4043724eb98b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:07.235148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f448db2-bdd8-4183-a8a5-4043724eb98b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:07.237927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f448db2-bdd8-4183-a8a5-4043724eb98b service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:07.253159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1361/2735] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:31:08.992997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145d702-961c-48f0-a14a-0ed72e988885 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:08.995423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-1145d702-961c-48f0-a14a-0ed72e988885 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:09.099508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:09.139803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1145d702-961c-48f0-a14a-0ed72e988885 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:09.148167 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c57dc-b52a-4c3f-854f-df383b1648e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:09.148841 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c57dc-b52a-4c3f-854f-df383b1648e8 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:09.149155 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c57dc-b52a-4c3f-854f-df383b1648e8 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:09.149431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8c57dc-b52a-4c3f-854f-df383b1648e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:09.152122 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0b8c57dc-b52a-4c3f-854f-df383b1648e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:09.168511 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c57dc-b52a-4c3f-854f-df383b1648e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:09.170244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8c57dc-b52a-4c3f-854f-df383b1648e8 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:09.182761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1375/2736] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:31:11.048598 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f448db2-bdd8-4183-a8a5-4043724eb98b service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:11.050511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0f448db2-bdd8-4183-a8a5-4043724eb98b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:11.064295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f448db2-bdd8-4183-a8a5-4043724eb98b service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:11.072232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8331e0ce-f1ff-4bd6-a468-c7aaf7b8b290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:11.073083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331e0ce-f1ff-4bd6-a468-c7aaf7b8b290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:11.073360 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331e0ce-f1ff-4bd6-a468-c7aaf7b8b290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:11.073659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8331e0ce-f1ff-4bd6-a468-c7aaf7b8b290 admin admin] PATH_INFO: `/v3/users/617ba5f9896b457f8182f760ed44aa58` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:11.076333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331e0ce-f1ff-4bd6-a468-c7aaf7b8b290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=617ba5f9896b457f8182f760ed44aa58)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:11.079535 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8331e0ce-f1ff-4bd6-a468-c7aaf7b8b290 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:11.118890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:11.159041 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c57dc-b52a-4c3f-854f-df383b1648e8 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:11.167660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8331e0ce-f1ff-4bd6-a468-c7aaf7b8b290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '617ba5f9896b457f8182f760ed44aa58'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.173462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8c57dc-b52a-4c3f-854f-df383b1648e8 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:11.175580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8331e0ce-f1ff-4bd6-a468-c7aaf7b8b290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '617ba5f9896b457f8182f760ed44aa58'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.178687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:11.179334 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:11.179638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:11.179882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] PATH_INFO: `/v3/users/0075be376b3e497ba1574d54043315f3` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:11.181092 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=0075be376b3e497ba1574d54043315f3)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:11.183281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:11.190280 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8331e0ce-f1ff-4bd6-a468-c7aaf7b8b290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '617ba5f9896b457f8182f760ed44aa58'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.191118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8331e0ce-f1ff-4bd6-a468-c7aaf7b8b290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '617ba5f9896b457f8182f760ed44aa58'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.200486 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8331e0ce-f1ff-4bd6-a468-c7aaf7b8b290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '617ba5f9896b457f8182f760ed44aa58'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.210206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1362/2737] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:31:11 2022] DELETE /identity/v3/users/617ba5f9896b457f8182f760ed44aa58 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:11.240626 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0075be376b3e497ba1574d54043315f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.251464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0075be376b3e497ba1574d54043315f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.252106 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0075be376b3e497ba1574d54043315f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.265613 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0075be376b3e497ba1574d54043315f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.286122 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0075be376b3e497ba1574d54043315f3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.292980 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1376/2738] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:31:11 2022] DELETE /identity/v3/users/0075be376b3e497ba1574d54043315f3 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:11.297592 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51aa9c-9489-4056-8614-af3be5161015 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:11.297984 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51aa9c-9489-4056-8614-af3be5161015 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:11.298331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51aa9c-9489-4056-8614-af3be5161015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:11.319562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7a51aa9c-9489-4056-8614-af3be5161015 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:11.320834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7a51aa9c-9489-4056-8614-af3be5161015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:11.409736 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1363/2739] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:31:11.420276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:11.421717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:11.437945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51aa9c-9489-4056-8614-af3be5161015 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:11.439459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7a51aa9c-9489-4056-8614-af3be5161015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:11.452250 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51aa9c-9489-4056-8614-af3be5161015 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:11.457556 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:11.458315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:11.458659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:11.459018 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025 admin admin] PATH_INFO: `/v3/projects/0d61f0fe37cf4dc0ba6a2415ddf14518` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:11.460960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d61f0fe37cf4dc0ba6a2415ddf14518)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:11.463536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:11.505751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69d09f-3e3b-441a-b9a4-8b42be9ae3fe admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:11.515895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142d4776-2969-402a-84f8-f3376e2b2595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:11.517075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d4776-2969-402a-84f8-f3376e2b2595 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:11.517340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d4776-2969-402a-84f8-f3376e2b2595 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:11.517500 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142d4776-2969-402a-84f8-f3376e2b2595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:11.528032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d4776-2969-402a-84f8-f3376e2b2595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:11.529486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142d4776-2969-402a-84f8-f3376e2b2595 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:11.533764 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d61f0fe37cf4dc0ba6a2415ddf14518', 'request_id': 'req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.541017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1377/2740] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:31:11.541742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d61f0fe37cf4dc0ba6a2415ddf14518', 'request_id': 'req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.547752 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d61f0fe37cf4dc0ba6a2415ddf14518', 'request_id': 'req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.562416 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025 admin admin] The token cache is being invalidate because project 0d61f0fe37cf4dc0ba6a2415ddf14518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:31:11.562713 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.563398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a8f608f7-0fa4-4654-a1ed-fa1a84d24025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:11.564925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1364/2741] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:31:11 2022] DELETE /identity/v3/projects/0d61f0fe37cf4dc0ba6a2415ddf14518 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:31:11.569939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd60671-1e66-40b4-9ccb-7c44942120ff None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:11.570341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd60671-1e66-40b4-9ccb-7c44942120ff None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:11.571314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd60671-1e66-40b4-9ccb-7c44942120ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:11.595404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-6dd60671-1e66-40b4-9ccb-7c44942120ff None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:11.597050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6dd60671-1e66-40b4-9ccb-7c44942120ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:11.661607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:11.677504 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1378/2742] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:11.685495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86595d4d-35e6-4719-9b39-b4593a9e3aab None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:11.685890 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86595d4d-35e6-4719-9b39-b4593a9e3aab None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:11.686870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86595d4d-35e6-4719-9b39-b4593a9e3aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:11.716237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-86595d4d-35e6-4719-9b39-b4593a9e3aab None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:11.717574 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-86595d4d-35e6-4719-9b39-b4593a9e3aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:11.726468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1365/2743] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:11.730900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd60671-1e66-40b4-9ccb-7c44942120ff None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:11.741161 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd60671-1e66-40b4-9ccb-7c44942120ff None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:11.746633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a300d-505e-4504-ad65-2da1733b92ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:11.747168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a300d-505e-4504-ad65-2da1733b92ba admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:11.747444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a300d-505e-4504-ad65-2da1733b92ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:11.747717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501a300d-505e-4504-ad65-2da1733b92ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:11.748378 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a300d-505e-4504-ad65-2da1733b92ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:31:11.748683 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a300d-505e-4504-ad65-2da1733b92ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:11.750383 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501a300d-505e-4504-ad65-2da1733b92ba admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:11.767368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1379/2744] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:31:11.768496 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86595d4d-35e6-4719-9b39-b4593a9e3aab None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:11.780818 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86595d4d-35e6-4719-9b39-b4593a9e3aab None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:11.786377 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15dae99-0778-4043-95f2-8d9f67fb6ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:11.787024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dae99-0778-4043-95f2-8d9f67fb6ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:11.787307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dae99-0778-4043-95f2-8d9f67fb6ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:11.787584 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15dae99-0778-4043-95f2-8d9f67fb6ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:11.788230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15dae99-0778-4043-95f2-8d9f67fb6ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:11.790216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15dae99-0778-4043-95f2-8d9f67fb6ed0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:11.802350 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:11.837481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1366/2745] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:31:11 2022] POST /identity/v3/projects => generated 383 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:31:11.839094 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a300d-505e-4504-ad65-2da1733b92ba admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:11.850553 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501a300d-505e-4504-ad65-2da1733b92ba admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:11.875387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15dae99-0778-4043-95f2-8d9f67fb6ed0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:11.878320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b15dae99-0778-4043-95f2-8d9f67fb6ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:11.930241 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbcb5f-01dc-4acd-8bd6-fb59a4071742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:11.930872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbcb5f-01dc-4acd-8bd6-fb59a4071742 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:11.931193 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbcb5f-01dc-4acd-8bd6-fb59a4071742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:11.931470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dbcb5f-01dc-4acd-8bd6-fb59a4071742 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:11.932115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbcb5f-01dc-4acd-8bd6-fb59a4071742 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:11.934062 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dbcb5f-01dc-4acd-8bd6-fb59a4071742 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:11.966175 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1380/2746] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:31:11 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:31:11.970853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbcb5f-01dc-4acd-8bd6-fb59a4071742 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:11.978770 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15dae99-0778-4043-95f2-8d9f67fb6ed0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:11.983807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:11.984607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:11.984779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dbcb5f-01dc-4acd-8bd6-fb59a4071742 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:11.985006 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:11.985349 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] PATH_INFO: `/v3/projects/927fcbfca61049018c42eb9cda6f8f48` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:11.986639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=927fcbfca61049018c42eb9cda6f8f48)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:11.988857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:11.990682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b2377-aaaa-45fc-9cbc-c73b2af97763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:11.991832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b2377-aaaa-45fc-9cbc-c73b2af97763 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:11.992186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b2377-aaaa-45fc-9cbc-c73b2af97763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:11.992529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b2377-aaaa-45fc-9cbc-c73b2af97763 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:11.993336 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b2377-aaaa-45fc-9cbc-c73b2af97763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:11.995596 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b2377-aaaa-45fc-9cbc-c73b2af97763 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:12.017775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1381/2747] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:31:12.021708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b2377-aaaa-45fc-9cbc-c73b2af97763 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:12.035671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b2377-aaaa-45fc-9cbc-c73b2af97763 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:12.054648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '927fcbfca61049018c42eb9cda6f8f48', 'request_id': 'req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.064533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '927fcbfca61049018c42eb9cda6f8f48', 'request_id': 'req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.073356 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '927fcbfca61049018c42eb9cda6f8f48', 'request_id': 'req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.086872 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] The token cache is being invalidate because project 927fcbfca61049018c42eb9cda6f8f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:31:12.087841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.088804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.090691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1367/2748] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:31:11 2022] DELETE /identity/v3/projects/927fcbfca61049018c42eb9cda6f8f48 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:12.095161 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:12.096881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:12.111861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b5fb1-abc1-4b1f-b451-a523068d4e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:12.113230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b5fb1-abc1-4b1f-b451-a523068d4e02 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:12.113230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b5fb1-abc1-4b1f-b451-a523068d4e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:12.113503 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b5fb1-abc1-4b1f-b451-a523068d4e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:12.114585 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b5fb1-abc1-4b1f-b451-a523068d4e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:12.116600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b5fb1-abc1-4b1f-b451-a523068d4e02 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:12.134022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1382/2749] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:31:12.135794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b5fb1-abc1-4b1f-b451-a523068d4e02 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:12.137776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fd0b5fb1-abc1-4b1f-b451-a523068d4e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:12.154814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b5fb1-abc1-4b1f-b451-a523068d4e02 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:12.162443 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126175f-3e7f-444b-a31d-0f42ad397887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:12.163312 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126175f-3e7f-444b-a31d-0f42ad397887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:12.163698 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126175f-3e7f-444b-a31d-0f42ad397887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:12.164091 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126175f-3e7f-444b-a31d-0f42ad397887 admin admin] PATH_INFO: `/v3/projects/499e128cf94a416ab22578156360dfc4/users/ac85cd4dd183482c96fc7e5ea4003807/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:12.176155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5e01ef-90b9-4bc7-acb7-68ebcde49076 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:12.183069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:12.183846 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:12.184085 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126175f-3e7f-444b-a31d-0f42ad397887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=499e128cf94a416ab22578156360dfc4, user_id=ac85cd4dd183482c96fc7e5ea4003807, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:12.184548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:12.184903 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] PATH_INFO: `/v3/users/6a47ddf6d17443ff8ca9b25c41107ea9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:12.186235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a47ddf6d17443ff8ca9b25c41107ea9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:12.187146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126175f-3e7f-444b-a31d-0f42ad397887 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:12.188345 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:12.207605 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1383/2750] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:31:12 2022] PUT /identity/v3/projects/499e128cf94a416ab22578156360dfc4/users/ac85cd4dd183482c96fc7e5ea4003807/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:31:12.217027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:12.219292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e20288-fbf2-442f-ab15-51ec7c799884 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:12.219458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e20288-fbf2-442f-ab15-51ec7c799884 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:12.219778 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e20288-fbf2-442f-ab15-51ec7c799884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:12.255201 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a47ddf6d17443ff8ca9b25c41107ea9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.261970 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-33e20288-fbf2-442f-ab15-51ec7c799884 None None] MFA Rules not processed for user `ac85cd4dd183482c96fc7e5ea4003807`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:12.263633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-33e20288-fbf2-442f-ab15-51ec7c799884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:12.265433 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a47ddf6d17443ff8ca9b25c41107ea9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.285566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a47ddf6d17443ff8ca9b25c41107ea9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.287479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a47ddf6d17443ff8ca9b25c41107ea9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.289963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:12.298397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a47ddf6d17443ff8ca9b25c41107ea9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.312112 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1368/2751] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:31:12 2022] DELETE /identity/v3/users/6a47ddf6d17443ff8ca9b25c41107ea9 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:31:12.372007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1384/2752] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:31:12.386122 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:12.390362 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:12.581174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c093f-87ea-4aef-99c1-f51546605290 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:12.587429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b712e56-d0de-4ba2-bb23-355e56153ef9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:12.588260 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b712e56-d0de-4ba2-bb23-355e56153ef9 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:12.588547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b712e56-d0de-4ba2-bb23-355e56153ef9 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:12.588832 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b712e56-d0de-4ba2-bb23-355e56153ef9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:12.654420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e20288-fbf2-442f-ab15-51ec7c799884 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:12.655977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-33e20288-fbf2-442f-ab15-51ec7c799884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:12.657123 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b712e56-d0de-4ba2-bb23-355e56153ef9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:12.659168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b712e56-d0de-4ba2-bb23-355e56153ef9 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:12.679167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1369/2753] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:31:12.731352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e20288-fbf2-442f-ab15-51ec7c799884 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:12.795187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:12.795831 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:12.796110 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:12.796386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] PATH_INFO: `/v3/projects/050233c3bd1e48f081ef450ba21f546d` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:12.797550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] RBAC: Authorizing `identity:delete_project(project_id=050233c3bd1e48f081ef450ba21f546d)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:12.799490 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:12.856189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '050233c3bd1e48f081ef450ba21f546d', 'request_id': 'req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.864836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '050233c3bd1e48f081ef450ba21f546d', 'request_id': 'req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.872641 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '050233c3bd1e48f081ef450ba21f546d', 'request_id': 'req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.880050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:12.886101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] The token cache is being invalidate because project 050233c3bd1e48f081ef450ba21f546d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:31:12.886101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.886666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:12.888495 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1385/2754] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:31:12 2022] DELETE /identity/v3/projects/050233c3bd1e48f081ef450ba21f546d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:31:12.892903 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b712e56-d0de-4ba2-bb23-355e56153ef9 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:12.904357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b712e56-d0de-4ba2-bb23-355e56153ef9 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:12.909751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:12.976015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:12.976735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:12.977022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:12.977297 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] PATH_INFO: `/v3/users/3244b3fb7f9045a8a137e0ca3ddb6ab2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:12.979082 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3244b3fb7f9045a8a137e0ca3ddb6ab2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:12.980870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:13.041607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3244b3fb7f9045a8a137e0ca3ddb6ab2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:13.051862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3244b3fb7f9045a8a137e0ca3ddb6ab2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:13.077720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3244b3fb7f9045a8a137e0ca3ddb6ab2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:13.079741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3244b3fb7f9045a8a137e0ca3ddb6ab2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:13.096693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3244b3fb7f9045a8a137e0ca3ddb6ab2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:13.108254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1370/2755] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:31:12 2022] DELETE /identity/v3/users/3244b3fb7f9045a8a137e0ca3ddb6ab2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:13.414234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:13.415892 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:13.431584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39bcf1-1c76-46a6-aa2c-085e56f0a07c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:13.511186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f66597a-4e90-4650-bf4f-93749fe188d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:13.511954 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66597a-4e90-4650-bf4f-93749fe188d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:13.512229 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66597a-4e90-4650-bf4f-93749fe188d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:13.512510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f66597a-4e90-4650-bf4f-93749fe188d3 admin admin] PATH_INFO: `/v3/projects/1a0823386d2f4c74b15f421fd1608fa9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:13.513670 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66597a-4e90-4650-bf4f-93749fe188d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a0823386d2f4c74b15f421fd1608fa9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:13.515895 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f66597a-4e90-4650-bf4f-93749fe188d3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:13.573211 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8f66597a-4e90-4650-bf4f-93749fe188d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a0823386d2f4c74b15f421fd1608fa9', 'request_id': 'req-8f66597a-4e90-4650-bf4f-93749fe188d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:13.582584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8f66597a-4e90-4650-bf4f-93749fe188d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a0823386d2f4c74b15f421fd1608fa9', 'request_id': 'req-8f66597a-4e90-4650-bf4f-93749fe188d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:13.590452 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8f66597a-4e90-4650-bf4f-93749fe188d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a0823386d2f4c74b15f421fd1608fa9', 'request_id': 'req-8f66597a-4e90-4650-bf4f-93749fe188d3'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:13.603772 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8f66597a-4e90-4650-bf4f-93749fe188d3 admin admin] The token cache is being invalidate because project 1a0823386d2f4c74b15f421fd1608fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:31:13.604222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8f66597a-4e90-4650-bf4f-93749fe188d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:13.604981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-8f66597a-4e90-4650-bf4f-93749fe188d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:13.606643 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1386/2756] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:31:13 2022] DELETE /identity/v3/projects/1a0823386d2f4c74b15f421fd1608fa9 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:23.104553 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=49,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:23.600019 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:25.427115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:25.427115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:25.449200 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28851da6-6aa5-464e-b588-510a1fa52ee9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:25.533835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:25.534458 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:25.534784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:25.535103 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] PATH_INFO: `/v3/users/ac85cd4dd183482c96fc7e5ea4003807` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:25.536405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac85cd4dd183482c96fc7e5ea4003807)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:25.538703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:25.596187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac85cd4dd183482c96fc7e5ea4003807'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:25.606682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac85cd4dd183482c96fc7e5ea4003807'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:25.619983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac85cd4dd183482c96fc7e5ea4003807'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:25.621028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac85cd4dd183482c96fc7e5ea4003807'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:25.628904 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac85cd4dd183482c96fc7e5ea4003807'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:25.636324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1371/2757] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:31:25 2022] DELETE /identity/v3/users/ac85cd4dd183482c96fc7e5ea4003807 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:25.641463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7b628-4256-4752-83cd-958b709e17c3 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:25.642132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7b628-4256-4752-83cd-958b709e17c3 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:25.642132 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7b628-4256-4752-83cd-958b709e17c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:25.663046 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-60f7b628-4256-4752-83cd-958b709e17c3 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:25.665101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-60f7b628-4256-4752-83cd-958b709e17c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:25.751611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1387/2758] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:25.762191 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:25.763652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:25.823801 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4b57b-69d8-4ed4-bed1-0459e9281e8f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:25.827553 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9f107-dbb4-42a2-ba78-55c46ac3e6bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:25.828166 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9f107-dbb4-42a2-ba78-55c46ac3e6bd service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:25.828449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9f107-dbb4-42a2-ba78-55c46ac3e6bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:25.828718 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9f107-dbb4-42a2-ba78-55c46ac3e6bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:25.839207 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9f107-dbb4-42a2-ba78-55c46ac3e6bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:25.840775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9f107-dbb4-42a2-ba78-55c46ac3e6bd service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:25.851279 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1372/2759] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:31:26.224050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7b628-4256-4752-83cd-958b709e17c3 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:26.225513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-60f7b628-4256-4752-83cd-958b709e17c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:26.242835 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7b628-4256-4752-83cd-958b709e17c3 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:26.251532 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:26.252460 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:26.252755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:26.253045 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0 admin admin] PATH_INFO: `/v3/projects/499e128cf94a416ab22578156360dfc4` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:26.254312 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=499e128cf94a416ab22578156360dfc4)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:26.256241 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:26.319780 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '499e128cf94a416ab22578156360dfc4', 'request_id': 'req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:26.331882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '499e128cf94a416ab22578156360dfc4', 'request_id': 'req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:26.340651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '499e128cf94a416ab22578156360dfc4', 'request_id': 'req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:26.355081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0 admin admin] The token cache is being invalidate because project 499e128cf94a416ab22578156360dfc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:31:26.355474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:26.356407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-4918a0b1-d8a8-45ed-9aec-5f28e4154cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:26.358446 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1388/2760] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:31:26 2022] DELETE /identity/v3/projects/499e128cf94a416ab22578156360dfc4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:26.365163 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291c4d3-21db-4544-8bac-ea15cab5de35 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:26.365639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291c4d3-21db-4544-8bac-ea15cab5de35 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:26.366148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291c4d3-21db-4544-8bac-ea15cab5de35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:26.396020 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4291c4d3-21db-4544-8bac-ea15cab5de35 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:26.397993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4291c4d3-21db-4544-8bac-ea15cab5de35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:26.425635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:26.483909 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1373/2761] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:26.500057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd34f7-1ebd-400f-b4ef-b2473fea22f7 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:26.500394 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd34f7-1ebd-400f-b4ef-b2473fea22f7 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:26.500765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd34f7-1ebd-400f-b4ef-b2473fea22f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:26.532109 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-48fd34f7-1ebd-400f-b4ef-b2473fea22f7 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:26.534179 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-48fd34f7-1ebd-400f-b4ef-b2473fea22f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:26.547623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1389/2762] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:26.554746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291c4d3-21db-4544-8bac-ea15cab5de35 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:26.574788 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291c4d3-21db-4544-8bac-ea15cab5de35 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:26.584529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff4b1f-a60a-47e9-84fd-72387daadeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:26.585150 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff4b1f-a60a-47e9-84fd-72387daadeb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:26.585433 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff4b1f-a60a-47e9-84fd-72387daadeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:26.585771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff4b1f-a60a-47e9-84fd-72387daadeb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:26.586420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff4b1f-a60a-47e9-84fd-72387daadeb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:31:26.586727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff4b1f-a60a-47e9-84fd-72387daadeb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:26.588336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff4b1f-a60a-47e9-84fd-72387daadeb6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:26.607242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1374/2763] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:31:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:31:26.611665 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd34f7-1ebd-400f-b4ef-b2473fea22f7 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:26.621790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd34f7-1ebd-400f-b4ef-b2473fea22f7 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:26.626710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a30ea-15de-4ee2-a0d2-8a2ba1efb41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:26.627332 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a30ea-15de-4ee2-a0d2-8a2ba1efb41a admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:26.627607 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a30ea-15de-4ee2-a0d2-8a2ba1efb41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:26.627882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a30ea-15de-4ee2-a0d2-8a2ba1efb41a admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:26.628525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a30ea-15de-4ee2-a0d2-8a2ba1efb41a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:26.630498 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a30ea-15de-4ee2-a0d2-8a2ba1efb41a admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:26.662666 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1390/2764] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:31:26 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:31:26.667622 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff4b1f-a60a-47e9-84fd-72387daadeb6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:26.678129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff4b1f-a60a-47e9-84fd-72387daadeb6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:26.744072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b883cd-f026-4c1d-ae68-4106e5df47e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:26.744779 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b883cd-f026-4c1d-ae68-4106e5df47e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:26.745726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b883cd-f026-4c1d-ae68-4106e5df47e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:26.746126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b883cd-f026-4c1d-ae68-4106e5df47e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:26.746854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b883cd-f026-4c1d-ae68-4106e5df47e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:26.749110 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b883cd-f026-4c1d-ae68-4106e5df47e7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:26.774229 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1375/2765] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:31:26 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:31:26.779658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a30ea-15de-4ee2-a0d2-8a2ba1efb41a admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:26.792678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a30ea-15de-4ee2-a0d2-8a2ba1efb41a admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:26.798439 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbe101-9737-4acf-9a36-6825ec759b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:26.799050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbe101-9737-4acf-9a36-6825ec759b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:26.799331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbe101-9737-4acf-9a36-6825ec759b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:26.799593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbe101-9737-4acf-9a36-6825ec759b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:26.800247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbe101-9737-4acf-9a36-6825ec759b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:26.801823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbe101-9737-4acf-9a36-6825ec759b78 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:26.813035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1391/2766] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:31:26.818037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b883cd-f026-4c1d-ae68-4106e5df47e7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:26.834073 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b883cd-f026-4c1d-ae68-4106e5df47e7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:26.840594 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9468f-2429-46e5-ad7f-53e5d96c915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:26.841355 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9468f-2429-46e5-ad7f-53e5d96c915e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:26.841644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9468f-2429-46e5-ad7f-53e5d96c915e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:26.841919 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9468f-2429-46e5-ad7f-53e5d96c915e admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:26.842604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9468f-2429-46e5-ad7f-53e5d96c915e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:26.844386 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9468f-2429-46e5-ad7f-53e5d96c915e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:26.858168 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1376/2767] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:31:26.863012 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbe101-9737-4acf-9a36-6825ec759b78 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:26.879704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbe101-9737-4acf-9a36-6825ec759b78 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:26.884934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abb3b1-5d41-4fd6-9972-62077e81d433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:26.885718 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abb3b1-5d41-4fd6-9972-62077e81d433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:26.886029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abb3b1-5d41-4fd6-9972-62077e81d433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:26.886308 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abb3b1-5d41-4fd6-9972-62077e81d433 admin admin] PATH_INFO: `/v3/projects/e3a1f592f97b4f6ea6248126da5f977e/users/f208b73b93be40f48c6173ed8a59680b/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:26.903964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abb3b1-5d41-4fd6-9972-62077e81d433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a1f592f97b4f6ea6248126da5f977e, user_id=f208b73b93be40f48c6173ed8a59680b, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:26.907135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abb3b1-5d41-4fd6-9972-62077e81d433 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:26.924874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1392/2768] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:31:26 2022] PUT /identity/v3/projects/e3a1f592f97b4f6ea6248126da5f977e/users/f208b73b93be40f48c6173ed8a59680b/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:26.937265 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5154a66-0106-475b-98bf-f56573bd7ec0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:26.937593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5154a66-0106-475b-98bf-f56573bd7ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:26.937825 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5154a66-0106-475b-98bf-f56573bd7ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:26.977777 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f5154a66-0106-475b-98bf-f56573bd7ec0 None None] MFA Rules not processed for user `f208b73b93be40f48c6173ed8a59680b`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:26.982035 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f5154a66-0106-475b-98bf-f56573bd7ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:27.064174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1377/2769] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:31:27.070522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abb3b1-5d41-4fd6-9972-62077e81d433 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:27.105547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:27.183976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abb3b1-5d41-4fd6-9972-62077e81d433 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:27.189905 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23060a35-690c-4d58-8b25-8b758fde20a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:27.191014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23060a35-690c-4d58-8b25-8b758fde20a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:27.193823 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23060a35-690c-4d58-8b25-8b758fde20a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:27.196070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23060a35-690c-4d58-8b25-8b758fde20a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:27.197653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23060a35-690c-4d58-8b25-8b758fde20a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:27.201272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23060a35-690c-4d58-8b25-8b758fde20a9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:27.265119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1393/2770] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:31:27 2022] POST /identity/v3/projects => generated 369 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:31:27.270424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5154a66-0106-475b-98bf-f56573bd7ec0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:27.284659 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5154a66-0106-475b-98bf-f56573bd7ec0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:27.299714 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:27.377648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d020a2a-37bb-4271-8444-9a020d6baaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:27.378463 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d020a2a-37bb-4271-8444-9a020d6baaa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:27.379041 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d020a2a-37bb-4271-8444-9a020d6baaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:27.379497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d020a2a-37bb-4271-8444-9a020d6baaa9 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:27.380571 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d020a2a-37bb-4271-8444-9a020d6baaa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:27.384460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d020a2a-37bb-4271-8444-9a020d6baaa9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:27.422026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1378/2771] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:31:27 2022] POST /identity/v3/users => generated 291 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:31:27.424356 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23060a35-690c-4d58-8b25-8b758fde20a9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:27.442015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23060a35-690c-4d58-8b25-8b758fde20a9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:27.449190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d6505-d9ab-4657-ab7d-a0a6a4b4e0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:27.450270 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d6505-d9ab-4657-ab7d-a0a6a4b4e0ce admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:27.450707 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d6505-d9ab-4657-ab7d-a0a6a4b4e0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:27.451176 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d6505-d9ab-4657-ab7d-a0a6a4b4e0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:27.452258 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d6505-d9ab-4657-ab7d-a0a6a4b4e0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:27.453898 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d6505-d9ab-4657-ab7d-a0a6a4b4e0ce admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:27.466418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1394/2772] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:31:27.471381 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d020a2a-37bb-4271-8444-9a020d6baaa9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:27.483500 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d020a2a-37bb-4271-8444-9a020d6baaa9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:27.489119 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea965d3-2891-4bc4-85fc-7d81fc82578a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:27.489790 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea965d3-2891-4bc4-85fc-7d81fc82578a admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:27.490204 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea965d3-2891-4bc4-85fc-7d81fc82578a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:27.490365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea965d3-2891-4bc4-85fc-7d81fc82578a admin admin] PATH_INFO: `/v3/projects/60c23476448c4ce7acb296001baecc2c/users/93c4d1af93d547559538eda4736346a2/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:27.508165 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea965d3-2891-4bc4-85fc-7d81fc82578a admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c23476448c4ce7acb296001baecc2c, user_id=93c4d1af93d547559538eda4736346a2, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:27.511138 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea965d3-2891-4bc4-85fc-7d81fc82578a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:27.531577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1379/2773] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:31:27 2022] PUT /identity/v3/projects/60c23476448c4ce7acb296001baecc2c/users/93c4d1af93d547559538eda4736346a2/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:27.536174 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d6505-d9ab-4657-ab7d-a0a6a4b4e0ce admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:27.547807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d6505-d9ab-4657-ab7d-a0a6a4b4e0ce admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:27.612225 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af24c0d-4f98-4802-84b2-a8e562a84bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:27.612966 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af24c0d-4f98-4802-84b2-a8e562a84bcf admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:27.613130 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af24c0d-4f98-4802-84b2-a8e562a84bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:27.613379 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af24c0d-4f98-4802-84b2-a8e562a84bcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:27.614091 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af24c0d-4f98-4802-84b2-a8e562a84bcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:27.615656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af24c0d-4f98-4802-84b2-a8e562a84bcf admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:27.626932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1395/2774] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:31:27.633638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea965d3-2891-4bc4-85fc-7d81fc82578a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:27.647514 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea965d3-2891-4bc4-85fc-7d81fc82578a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:27.656127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadb396-7fe3-4595-8d09-11cdaadc9ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:27.657218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb396-7fe3-4595-8d09-11cdaadc9ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:27.657666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb396-7fe3-4595-8d09-11cdaadc9ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:27.658167 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadb396-7fe3-4595-8d09-11cdaadc9ee4 admin admin] PATH_INFO: `/v3/projects/60c23476448c4ce7acb296001baecc2c/users/93c4d1af93d547559538eda4736346a2/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:27.661449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadb396-7fe3-4595-8d09-11cdaadc9ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c23476448c4ce7acb296001baecc2c, user_id=93c4d1af93d547559538eda4736346a2, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:27.666586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadb396-7fe3-4595-8d09-11cdaadc9ee4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:27.681511 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1380/2775] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:31:27 2022] PUT /identity/v3/projects/60c23476448c4ce7acb296001baecc2c/users/93c4d1af93d547559538eda4736346a2/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:27.693430 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e44fc3-5773-45f7-96a3-51ebecc046eb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:27.693993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e44fc3-5773-45f7-96a3-51ebecc046eb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:27.693993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e44fc3-5773-45f7-96a3-51ebecc046eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:27.723747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-41e44fc3-5773-45f7-96a3-51ebecc046eb None None] MFA Rules not processed for user `93c4d1af93d547559538eda4736346a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:27.725003 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-41e44fc3-5773-45f7-96a3-51ebecc046eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:27.804227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1396/2776] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:27.823400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadb396-7fe3-4595-8d09-11cdaadc9ee4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:27.825924 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ccadb396-7fe3-4595-8d09-11cdaadc9ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:27.893440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadb396-7fe3-4595-8d09-11cdaadc9ee4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:27.897130 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d81438-1d95-4f32-b0f9-10da6ecd6c92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:27.897737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d81438-1d95-4f32-b0f9-10da6ecd6c92 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:27.898022 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d81438-1d95-4f32-b0f9-10da6ecd6c92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:27.898329 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d81438-1d95-4f32-b0f9-10da6ecd6c92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:27.909068 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d81438-1d95-4f32-b0f9-10da6ecd6c92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:27.910591 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d81438-1d95-4f32-b0f9-10da6ecd6c92 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:27.921319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1381/2777] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:31:28.013389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e44fc3-5773-45f7-96a3-51ebecc046eb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:28.028467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e44fc3-5773-45f7-96a3-51ebecc046eb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:28.032148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b755a1-b3a5-4ccc-b7c4-73038f1279c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:28.032768 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b755a1-b3a5-4ccc-b7c4-73038f1279c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:28.033044 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b755a1-b3a5-4ccc-b7c4-73038f1279c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:28.033311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b755a1-b3a5-4ccc-b7c4-73038f1279c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:28.046696 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:28.109363 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b755a1-b3a5-4ccc-b7c4-73038f1279c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:28.111286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b755a1-b3a5-4ccc-b7c4-73038f1279c7 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:28.123708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1397/2778] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:31:30.707249 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d81438-1d95-4f32-b0f9-10da6ecd6c92 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:30.708945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c6d81438-1d95-4f32-b0f9-10da6ecd6c92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:30.793547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d81438-1d95-4f32-b0f9-10da6ecd6c92 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:30.801010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:30.801869 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:30.802253 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:30.802713 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] PATH_INFO: `/v3/users/e8ecf5615659422abf80f05686c39e97` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:30.804086 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ecf5615659422abf80f05686c39e97)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:30.806644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:30.830895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:30.880033 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ecf5615659422abf80f05686c39e97'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:30.889189 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ecf5615659422abf80f05686c39e97'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:30.915663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ecf5615659422abf80f05686c39e97'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:30.917052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ecf5615659422abf80f05686c39e97'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:30.928875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ecf5615659422abf80f05686c39e97'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:30.936447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1382/2779] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:31:30 2022] DELETE /identity/v3/users/e8ecf5615659422abf80f05686c39e97 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:30.942761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae236847-cdea-409f-a1b5-e6c48f003a67 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:30.943501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae236847-cdea-409f-a1b5-e6c48f003a67 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:30.943501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae236847-cdea-409f-a1b5-e6c48f003a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:30.972831 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ae236847-cdea-409f-a1b5-e6c48f003a67 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:30.974224 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ae236847-cdea-409f-a1b5-e6c48f003a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:31.063849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1398/2780] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:31.078906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:31.081946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:31.169879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cdab2-c0ed-48a5-8d4f-326d9b2e886b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:31.176085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab176cce-afb4-4122-b716-9779039207df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:31.177276 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab176cce-afb4-4122-b716-9779039207df service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:31.177782 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab176cce-afb4-4122-b716-9779039207df service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:31.178324 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab176cce-afb4-4122-b716-9779039207df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:31.194375 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab176cce-afb4-4122-b716-9779039207df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:31.196451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab176cce-afb4-4122-b716-9779039207df service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:31.208412 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1383/2781] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:31:31.544534 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae236847-cdea-409f-a1b5-e6c48f003a67 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:31.546570 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ae236847-cdea-409f-a1b5-e6c48f003a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:31.564814 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae236847-cdea-409f-a1b5-e6c48f003a67 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:31.571678 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e11606e-e435-49dc-b184-d231afb217d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:31.572351 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11606e-e435-49dc-b184-d231afb217d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:31.573074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11606e-e435-49dc-b184-d231afb217d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:31.573074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e11606e-e435-49dc-b184-d231afb217d7 admin admin] PATH_INFO: `/v3/projects/80f7b40b069047cebea956cf792c0bb9` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:31.575423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e11606e-e435-49dc-b184-d231afb217d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80f7b40b069047cebea956cf792c0bb9)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:31.577437 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e11606e-e435-49dc-b184-d231afb217d7 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:31.641878 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3e11606e-e435-49dc-b184-d231afb217d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80f7b40b069047cebea956cf792c0bb9', 'request_id': 'req-3e11606e-e435-49dc-b184-d231afb217d7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:31.652008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3e11606e-e435-49dc-b184-d231afb217d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80f7b40b069047cebea956cf792c0bb9', 'request_id': 'req-3e11606e-e435-49dc-b184-d231afb217d7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:31.664362 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3e11606e-e435-49dc-b184-d231afb217d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80f7b40b069047cebea956cf792c0bb9', 'request_id': 'req-3e11606e-e435-49dc-b184-d231afb217d7'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:31.679923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3e11606e-e435-49dc-b184-d231afb217d7 admin admin] The token cache is being invalidate because project 80f7b40b069047cebea956cf792c0bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:31:31.680217 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3e11606e-e435-49dc-b184-d231afb217d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:31.680887 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-3e11606e-e435-49dc-b184-d231afb217d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:31:31.682439 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1399/2782] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:31:31 2022] DELETE /identity/v3/projects/80f7b40b069047cebea956cf792c0bb9 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 16:31:31.687750 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd2f19-0a0a-4396-a2ba-62ae738a892e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:31.688058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd2f19-0a0a-4396-a2ba-62ae738a892e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:31.688337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd2f19-0a0a-4396-a2ba-62ae738a892e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:31.711459 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1dfd2f19-0a0a-4396-a2ba-62ae738a892e None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:31.712700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1dfd2f19-0a0a-4396-a2ba-62ae738a892e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:31.790938 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1384/2783] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:31.799441 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6c7fc-b3f2-4d97-bab7-f4f4adc01f02 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:31.799752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6c7fc-b3f2-4d97-bab7-f4f4adc01f02 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:31.800030 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e6c7fc-b3f2-4d97-bab7-f4f4adc01f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:31.827528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-21e6c7fc-b3f2-4d97-bab7-f4f4adc01f02 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:31.829508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-21e6c7fc-b3f2-4d97-bab7-f4f4adc01f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:31.839040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1400/2784] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:31:31.845440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd2f19-0a0a-4396-a2ba-62ae738a892e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:31.855823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd2f19-0a0a-4396-a2ba-62ae738a892e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:31.860703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4685b-24e2-453e-a2fa-c00bd02cd24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:31.861332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4685b-24e2-453e-a2fa-c00bd02cd24f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:31.861610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4685b-24e2-453e-a2fa-c00bd02cd24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:31.861876 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4685b-24e2-453e-a2fa-c00bd02cd24f admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:31.862602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4685b-24e2-453e-a2fa-c00bd02cd24f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:31:31.862900 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4685b-24e2-453e-a2fa-c00bd02cd24f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:31.864482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4685b-24e2-453e-a2fa-c00bd02cd24f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:31.878656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1385/2785] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:31:31.885467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6c7fc-b3f2-4d97-bab7-f4f4adc01f02 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:31.899008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e6c7fc-b3f2-4d97-bab7-f4f4adc01f02 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:31.905617 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3ac7c-a766-43ea-b150-cb37a59189b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:31.906272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3ac7c-a766-43ea-b150-cb37a59189b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:31.906548 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3ac7c-a766-43ea-b150-cb37a59189b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:31.906818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3ac7c-a766-43ea-b150-cb37a59189b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:31.907458 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3ac7c-a766-43ea-b150-cb37a59189b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:31.910843 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3ac7c-a766-43ea-b150-cb37a59189b9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:31.952448 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1401/2786] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:31:31 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:31:31.957311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4685b-24e2-453e-a2fa-c00bd02cd24f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:31.975526 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4685b-24e2-453e-a2fa-c00bd02cd24f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:32.001566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:32.052093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b367a-cddb-48c3-9942-f0a8bbbb74f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:32.052710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b367a-cddb-48c3-9942-f0a8bbbb74f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:32.052993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b367a-cddb-48c3-9942-f0a8bbbb74f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:32.053267 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b367a-cddb-48c3-9942-f0a8bbbb74f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:32.053906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b367a-cddb-48c3-9942-f0a8bbbb74f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:32.056351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b367a-cddb-48c3-9942-f0a8bbbb74f3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:32.094240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1386/2787] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:31:31 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:31:32.098556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3ac7c-a766-43ea-b150-cb37a59189b9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:32.114730 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3ac7c-a766-43ea-b150-cb37a59189b9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:32.122642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e3cfc-50df-4063-94d9-99d168b992a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:32.123321 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e3cfc-50df-4063-94d9-99d168b992a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:32.123591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e3cfc-50df-4063-94d9-99d168b992a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:32.123858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e3cfc-50df-4063-94d9-99d168b992a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:32.124889 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e3cfc-50df-4063-94d9-99d168b992a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:32.126456 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e3cfc-50df-4063-94d9-99d168b992a8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:32.129140 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:32.141937 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1402/2788] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:31:32.146539 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b367a-cddb-48c3-9942-f0a8bbbb74f3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:32.158341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b367a-cddb-48c3-9942-f0a8bbbb74f3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:32.163477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef656e96-d879-46a9-be32-e2b003d50e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:32.164107 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef656e96-d879-46a9-be32-e2b003d50e51 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:32.164480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef656e96-d879-46a9-be32-e2b003d50e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:32.165713 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef656e96-d879-46a9-be32-e2b003d50e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:32.165713 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef656e96-d879-46a9-be32-e2b003d50e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:32.167724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef656e96-d879-46a9-be32-e2b003d50e51 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:32.179966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1387/2789] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:31:32.185470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e3cfc-50df-4063-94d9-99d168b992a8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:32.201104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e3cfc-50df-4063-94d9-99d168b992a8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:32.206694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd101e-9ff8-43ae-bd22-bcf8f2ea1c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:32.207502 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd101e-9ff8-43ae-bd22-bcf8f2ea1c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:32.207834 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd101e-9ff8-43ae-bd22-bcf8f2ea1c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:32.208161 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fd101e-9ff8-43ae-bd22-bcf8f2ea1c50 admin admin] PATH_INFO: `/v3/projects/5eb1e42761a74d7eb8509ac0f4d1c1db/users/10f2730695454358bc1707dd0815d880/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:32.226972 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd101e-9ff8-43ae-bd22-bcf8f2ea1c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb1e42761a74d7eb8509ac0f4d1c1db, user_id=10f2730695454358bc1707dd0815d880, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:32.229723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fd101e-9ff8-43ae-bd22-bcf8f2ea1c50 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:32.247039 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1403/2790] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:31:32 2022] PUT /identity/v3/projects/5eb1e42761a74d7eb8509ac0f4d1c1db/users/10f2730695454358bc1707dd0815d880/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:31:32.252445 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d631bc-09f0-481c-b5d0-c639e84bcf0c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:32.252723 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d631bc-09f0-481c-b5d0-c639e84bcf0c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:32.252997 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d631bc-09f0-481c-b5d0-c639e84bcf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:32.275266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a2d631bc-09f0-481c-b5d0-c639e84bcf0c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:32.276679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a2d631bc-09f0-481c-b5d0-c639e84bcf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:32.352733 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1388/2791] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:32.369923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd101e-9ff8-43ae-bd22-bcf8f2ea1c50 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:32.371863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-59fd101e-9ff8-43ae-bd22-bcf8f2ea1c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:32.435764 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fd101e-9ff8-43ae-bd22-bcf8f2ea1c50 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:32.439493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81ecea-b1d0-4ce2-bcc4-d2a38f588b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:32.440108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81ecea-b1d0-4ce2-bcc4-d2a38f588b84 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:32.440388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81ecea-b1d0-4ce2-bcc4-d2a38f588b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:32.440660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81ecea-b1d0-4ce2-bcc4-d2a38f588b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:32.451515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81ecea-b1d0-4ce2-bcc4-d2a38f588b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:32.452976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81ecea-b1d0-4ce2-bcc4-d2a38f588b84 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:32.463508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1404/2792] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:31:33.358155 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0fd01-0c76-4d30-8bb6-7e68d5fbb7a2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:33.358921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0fd01-0c76-4d30-8bb6-7e68d5fbb7a2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:33.358921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0fd01-0c76-4d30-8bb6-7e68d5fbb7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:33.389724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7be0fd01-0c76-4d30-8bb6-7e68d5fbb7a2 None None] MFA Rules not processed for user `10f2730695454358bc1707dd0815d880`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:33.391251 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7be0fd01-0c76-4d30-8bb6-7e68d5fbb7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:33.484795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1389/2793] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:33.497948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81ecea-b1d0-4ce2-bcc4-d2a38f588b84 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:31:33.500722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fc81ecea-b1d0-4ce2-bcc4-d2a38f588b84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:33.580240 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81ecea-b1d0-4ce2-bcc4-d2a38f588b84 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:31:33.584292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bba45-3937-4fc7-ada5-abc5875ec834 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:31:33.584920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bba45-3937-4fc7-ada5-abc5875ec834 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:33.585196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bba45-3937-4fc7-ada5-abc5875ec834 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:33.585465 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bba45-3937-4fc7-ada5-abc5875ec834 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:33.597104 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bba45-3937-4fc7-ada5-abc5875ec834 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:31:33.600921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bba45-3937-4fc7-ada5-abc5875ec834 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:31:33.623716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1405/2794] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:31:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:31:35.089589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdd876-ec52-4b5c-9a7d-f94c02b80e97 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:31:35.090420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdd876-ec52-4b5c-9a7d-f94c02b80e97 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:31:35.091269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdd876-ec52-4b5c-9a7d-f94c02b80e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:31:35.129251 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ebbdd876-ec52-4b5c-9a7d-f94c02b80e97 None None] MFA Rules not processed for user `10f2730695454358bc1707dd0815d880`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:31:35.130588 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ebbdd876-ec52-4b5c-9a7d-f94c02b80e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:31:35.139019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1390/2795] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:31:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:31:43.613013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=63,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:31:45.120617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:32:05.003569 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bba45-3937-4fc7-ada5-abc5875ec834 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:32:05.005625 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e06bba45-3937-4fc7-ada5-abc5875ec834 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:32:05.019563 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bba45-3937-4fc7-ada5-abc5875ec834 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:32:05.023665 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d0d24-9f52-46f6-a1a3-a79587c41067 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:32:05.024265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d0d24-9f52-46f6-a1a3-a79587c41067 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:32:05.024542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d0d24-9f52-46f6-a1a3-a79587c41067 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:32:05.024813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d0d24-9f52-46f6-a1a3-a79587c41067 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:32:05.026912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-673d0d24-9f52-46f6-a1a3-a79587c41067 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:32:05.097675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d0d24-9f52-46f6-a1a3-a79587c41067 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:32:05.099595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d0d24-9f52-46f6-a1a3-a79587c41067 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:32:05.110247 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1406/2796] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:32:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:32:15.105013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=28 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:34:08.718200 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdd876-ec52-4b5c-9a7d-f94c02b80e97 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:34:08.728791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdd876-ec52-4b5c-9a7d-f94c02b80e97 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:34:08.732273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df2a2e-20d6-4e9d-9241-a7d11ea33f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:34:08.732868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df2a2e-20d6-4e9d-9241-a7d11ea33f84 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:34:08.733148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df2a2e-20d6-4e9d-9241-a7d11ea33f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:34:08.733415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91df2a2e-20d6-4e9d-9241-a7d11ea33f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:34:08.735809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-91df2a2e-20d6-4e9d-9241-a7d11ea33f84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:34:08.745638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df2a2e-20d6-4e9d-9241-a7d11ea33f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:34:08.747011 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91df2a2e-20d6-4e9d-9241-a7d11ea33f84 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:34:08.755014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1391/2797] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:34:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:34:18.751404 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:36:13.984457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d0d24-9f52-46f6-a1a3-a79587c41067 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:36:13.985853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-673d0d24-9f52-46f6-a1a3-a79587c41067 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:36:14.053642 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d0d24-9f52-46f6-a1a3-a79587c41067 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:36:14.058631 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab4e70-b615-429b-addc-715092f1f998 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:36:14.059233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab4e70-b615-429b-addc-715092f1f998 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:36:14.059514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab4e70-b615-429b-addc-715092f1f998 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:36:14.059783 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab4e70-b615-429b-addc-715092f1f998 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:36:14.061466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-86ab4e70-b615-429b-addc-715092f1f998 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:36:14.073797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab4e70-b615-429b-addc-715092f1f998 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:36:14.075390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab4e70-b615-429b-addc-715092f1f998 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:36:14.085625 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1407/2798] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:36:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:36:24.081368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=32 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:36:27.283241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df2a2e-20d6-4e9d-9241-a7d11ea33f84 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:36:27.292656 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91df2a2e-20d6-4e9d-9241-a7d11ea33f84 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:36:27.296216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce241c89-c9e1-4da4-822a-4289ccc6908d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:36:27.296799 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce241c89-c9e1-4da4-822a-4289ccc6908d service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:36:27.297076 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce241c89-c9e1-4da4-822a-4289ccc6908d service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:36:27.297347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce241c89-c9e1-4da4-822a-4289ccc6908d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:36:27.299944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ce241c89-c9e1-4da4-822a-4289ccc6908d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:36:27.357616 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce241c89-c9e1-4da4-822a-4289ccc6908d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:36:27.358983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce241c89-c9e1-4da4-822a-4289ccc6908d service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:36:27.368179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1392/2799] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:36:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:36:32.904874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab4e70-b615-429b-addc-715092f1f998 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:36:32.915188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab4e70-b615-429b-addc-715092f1f998 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:36:32.918813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513770c6-e7b3-4798-ac20-67b3500856fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:36:32.919407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513770c6-e7b3-4798-ac20-67b3500856fc service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:36:32.919683 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513770c6-e7b3-4798-ac20-67b3500856fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:36:32.919955 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513770c6-e7b3-4798-ac20-67b3500856fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:36:32.929031 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513770c6-e7b3-4798-ac20-67b3500856fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:36:32.930492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513770c6-e7b3-4798-ac20-67b3500856fc service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:36:32.939432 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1408/2800] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:36:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:36:37.364342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=28 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:36:42.935245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=6 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:37:04.164281 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce241c89-c9e1-4da4-822a-4289ccc6908d service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:37:04.173761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce241c89-c9e1-4da4-822a-4289ccc6908d service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:37:04.177439 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe53f6-fd25-4158-a7d4-4e0a2ce8f548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:37:04.177936 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe53f6-fd25-4158-a7d4-4e0a2ce8f548 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:37:04.178242 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe53f6-fd25-4158-a7d4-4e0a2ce8f548 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:37:04.178517 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefe53f6-fd25-4158-a7d4-4e0a2ce8f548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:37:04.186886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe53f6-fd25-4158-a7d4-4e0a2ce8f548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:37:04.188296 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefe53f6-fd25-4158-a7d4-4e0a2ce8f548 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:37:04.215692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1393/2801] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:37:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:37:14.213126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=13 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:06.326444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513770c6-e7b3-4798-ac20-67b3500856fc service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:06.335191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513770c6-e7b3-4798-ac20-67b3500856fc service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:06.338914 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc921aa-13f4-47d4-8fcf-4103aa668b7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:06.339522 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc921aa-13f4-47d4-8fcf-4103aa668b7c service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:06.339786 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc921aa-13f4-47d4-8fcf-4103aa668b7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:06.340058 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc921aa-13f4-47d4-8fcf-4103aa668b7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:06.341772 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fbc921aa-13f4-47d4-8fcf-4103aa668b7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:06.408040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc921aa-13f4-47d4-8fcf-4103aa668b7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:06.409469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc921aa-13f4-47d4-8fcf-4103aa668b7c service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:06.418987 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1409/2802] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:38:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:38:06.802616 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe53f6-fd25-4158-a7d4-4e0a2ce8f548 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:06.804667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fefe53f6-fd25-4158-a7d4-4e0a2ce8f548 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:06.821083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefe53f6-fd25-4158-a7d4-4e0a2ce8f548 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:06.825895 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:06.826573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:06.826854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:06.827126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] PATH_INFO: `/v3/users/f208b73b93be40f48c6173ed8a59680b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:06.828261 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f208b73b93be40f48c6173ed8a59680b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:06.830209 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:06.877448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f208b73b93be40f48c6173ed8a59680b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:06.885143 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f208b73b93be40f48c6173ed8a59680b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:06.897239 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f208b73b93be40f48c6173ed8a59680b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:06.897881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f208b73b93be40f48c6173ed8a59680b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:06.904961 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f208b73b93be40f48c6173ed8a59680b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:06.910691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1394/2803] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:38:06 2022] DELETE /identity/v3/users/f208b73b93be40f48c6173ed8a59680b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:06.915719 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ad5c2-0eef-4ee0-a6a1-ae7c3ee56d1d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:06.916002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ad5c2-0eef-4ee0-a6a1-ae7c3ee56d1d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:06.916275 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ad5c2-0eef-4ee0-a6a1-ae7c3ee56d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:06.935812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f32ad5c2-0eef-4ee0-a6a1-ae7c3ee56d1d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:06.937083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f32ad5c2-0eef-4ee0-a6a1-ae7c3ee56d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:06.995606 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1410/2804] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:07.006719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:07.008145 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:07.064633 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5402e1e-1e91-443f-be23-dbf66b15a81d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:07.068215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d65ad-883a-403d-8c88-4122e8724e11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:07.068810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d65ad-883a-403d-8c88-4122e8724e11 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:07.069090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d65ad-883a-403d-8c88-4122e8724e11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:07.069363 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d65ad-883a-403d-8c88-4122e8724e11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:07.078642 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d65ad-883a-403d-8c88-4122e8724e11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:07.080061 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3d65ad-883a-403d-8c88-4122e8724e11 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:07.090400 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1395/2805] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:38:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:38:07.363170 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ad5c2-0eef-4ee0-a6a1-ae7c3ee56d1d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:07.364591 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f32ad5c2-0eef-4ee0-a6a1-ae7c3ee56d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:07.376309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ad5c2-0eef-4ee0-a6a1-ae7c3ee56d1d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:07.380997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:07.381632 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:07.381951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:07.382787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] PATH_INFO: `/v3/projects/e3a1f592f97b4f6ea6248126da5f977e` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:07.383915 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a1f592f97b4f6ea6248126da5f977e)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:07.385802 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:07.434434 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a1f592f97b4f6ea6248126da5f977e', 'request_id': 'req-875fd482-3d2a-445f-b32a-34744139e6df'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:07.441656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a1f592f97b4f6ea6248126da5f977e', 'request_id': 'req-875fd482-3d2a-445f-b32a-34744139e6df'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:07.451042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a1f592f97b4f6ea6248126da5f977e', 'request_id': 'req-875fd482-3d2a-445f-b32a-34744139e6df'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:07.463118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] The token cache is being invalidate because project e3a1f592f97b4f6ea6248126da5f977e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:38:07.463407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:07.464087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:07.465571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1411/2806] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:38:07 2022] DELETE /identity/v3/projects/e3a1f592f97b4f6ea6248126da5f977e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:07.469844 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d65ad-883a-403d-8c88-4122e8724e11 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:07.479693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d65ad-883a-403d-8c88-4122e8724e11 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:07.536703 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82998f97-1367-4dcc-aee7-a955d38557c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:07.537332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82998f97-1367-4dcc-aee7-a955d38557c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:07.537611 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82998f97-1367-4dcc-aee7-a955d38557c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:07.537886 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82998f97-1367-4dcc-aee7-a955d38557c1 admin admin] PATH_INFO: `/v3/users/93c4d1af93d547559538eda4736346a2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:07.539064 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82998f97-1367-4dcc-aee7-a955d38557c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c4d1af93d547559538eda4736346a2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:07.540993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82998f97-1367-4dcc-aee7-a955d38557c1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:07.557374 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:07.590624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82998f97-1367-4dcc-aee7-a955d38557c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c4d1af93d547559538eda4736346a2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:07.599524 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82998f97-1367-4dcc-aee7-a955d38557c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c4d1af93d547559538eda4736346a2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:07.612755 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82998f97-1367-4dcc-aee7-a955d38557c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c4d1af93d547559538eda4736346a2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:07.613448 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82998f97-1367-4dcc-aee7-a955d38557c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c4d1af93d547559538eda4736346a2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:07.624604 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82998f97-1367-4dcc-aee7-a955d38557c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c4d1af93d547559538eda4736346a2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:07.630490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1396/2807] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:38:07 2022] DELETE /identity/v3/users/93c4d1af93d547559538eda4736346a2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:07.906122 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:07.907583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:07.926558 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fd482-3d2a-445f-b32a-34744139e6df admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:07.940584 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:07.987470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385fe26-1087-4a63-9c7d-7990e7a34807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:07.988090 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385fe26-1087-4a63-9c7d-7990e7a34807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:07.988372 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385fe26-1087-4a63-9c7d-7990e7a34807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:07.988648 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385fe26-1087-4a63-9c7d-7990e7a34807 admin admin] PATH_INFO: `/v3/projects/60c23476448c4ce7acb296001baecc2c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:07.989781 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385fe26-1087-4a63-9c7d-7990e7a34807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60c23476448c4ce7acb296001baecc2c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:07.992241 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385fe26-1087-4a63-9c7d-7990e7a34807 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:08.037737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1385fe26-1087-4a63-9c7d-7990e7a34807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60c23476448c4ce7acb296001baecc2c', 'request_id': 'req-1385fe26-1087-4a63-9c7d-7990e7a34807'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:08.045395 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1385fe26-1087-4a63-9c7d-7990e7a34807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60c23476448c4ce7acb296001baecc2c', 'request_id': 'req-1385fe26-1087-4a63-9c7d-7990e7a34807'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:08.051687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1385fe26-1087-4a63-9c7d-7990e7a34807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60c23476448c4ce7acb296001baecc2c', 'request_id': 'req-1385fe26-1087-4a63-9c7d-7990e7a34807'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:08.062779 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1385fe26-1087-4a63-9c7d-7990e7a34807 admin admin] The token cache is being invalidate because project 60c23476448c4ce7acb296001baecc2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:38:08.063069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1385fe26-1087-4a63-9c7d-7990e7a34807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:08.063715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-1385fe26-1087-4a63-9c7d-7990e7a34807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:08.065204 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1412/2808] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:38:07 2022] DELETE /identity/v3/projects/60c23476448c4ce7acb296001baecc2c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:08.071747 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28937092-9a31-4451-bb7c-af364398925f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.072293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28937092-9a31-4451-bb7c-af364398925f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.072293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28937092-9a31-4451-bb7c-af364398925f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.092105 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-28937092-9a31-4451-bb7c-af364398925f None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:08.093305 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-28937092-9a31-4451-bb7c-af364398925f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:08.153293 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1397/2809] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:08.161276 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dcaa8-4478-4823-a828-325c4cb12acf None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.161593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dcaa8-4478-4823-a828-325c4cb12acf None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.161896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dcaa8-4478-4823-a828-325c4cb12acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.183818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-939dcaa8-4478-4823-a828-325c4cb12acf None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:08.185072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-939dcaa8-4478-4823-a828-325c4cb12acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:08.192994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1413/2810] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:08.197881 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28937092-9a31-4451-bb7c-af364398925f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:08.208405 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28937092-9a31-4451-bb7c-af364398925f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:08.213172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bf55a-7004-48ee-b56c-97acc8d7eaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:08.214113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bf55a-7004-48ee-b56c-97acc8d7eaf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.214398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bf55a-7004-48ee-b56c-97acc8d7eaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.214804 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bf55a-7004-48ee-b56c-97acc8d7eaf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.215532 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bf55a-7004-48ee-b56c-97acc8d7eaf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:38:08.215836 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bf55a-7004-48ee-b56c-97acc8d7eaf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:08.217586 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bf55a-7004-48ee-b56c-97acc8d7eaf2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:08.230179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1398/2811] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:38:08.234812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dcaa8-4478-4823-a828-325c4cb12acf None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:08.245341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dcaa8-4478-4823-a828-325c4cb12acf None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:08.249931 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb05ee-09e0-4a12-86d3-31bd8ace45b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:08.250582 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb05ee-09e0-4a12-86d3-31bd8ace45b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.250858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb05ee-09e0-4a12-86d3-31bd8ace45b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.251288 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb05ee-09e0-4a12-86d3-31bd8ace45b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.251931 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb05ee-09e0-4a12-86d3-31bd8ace45b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:08.253812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb05ee-09e0-4a12-86d3-31bd8ace45b3 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:08.280007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1414/2812] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:38:08 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:38:08.284230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bf55a-7004-48ee-b56c-97acc8d7eaf2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:08.294823 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bf55a-7004-48ee-b56c-97acc8d7eaf2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:08.353021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16c4ac-5852-4dd0-8744-b47baf77d5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:08.353624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16c4ac-5852-4dd0-8744-b47baf77d5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.353929 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16c4ac-5852-4dd0-8744-b47baf77d5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.354237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c16c4ac-5852-4dd0-8744-b47baf77d5d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.354875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16c4ac-5852-4dd0-8744-b47baf77d5d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:08.356773 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c16c4ac-5852-4dd0-8744-b47baf77d5d6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:08.379478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1399/2813] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:38:08 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:38:08.384461 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb05ee-09e0-4a12-86d3-31bd8ace45b3 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:08.394920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb05ee-09e0-4a12-86d3-31bd8ace45b3 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:08.399600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0217e98-1f36-44a6-a74c-57bad28b0e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:08.400128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0217e98-1f36-44a6-a74c-57bad28b0e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.400404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0217e98-1f36-44a6-a74c-57bad28b0e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.400672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0217e98-1f36-44a6-a74c-57bad28b0e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.401335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0217e98-1f36-44a6-a74c-57bad28b0e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:08.402940 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0217e98-1f36-44a6-a74c-57bad28b0e00 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:08.536583 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1415/2814] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 21 16:38:08.541481 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16c4ac-5852-4dd0-8744-b47baf77d5d6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:08.551521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c16c4ac-5852-4dd0-8744-b47baf77d5d6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:08.556192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b317e-631a-4e92-a428-a47529ef3421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:08.556805 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b317e-631a-4e92-a428-a47529ef3421 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.557084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b317e-631a-4e92-a428-a47529ef3421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.557357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b317e-631a-4e92-a428-a47529ef3421 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.558028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b317e-631a-4e92-a428-a47529ef3421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:08.559629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b317e-631a-4e92-a428-a47529ef3421 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:08.564999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:08.571169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1400/2815] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:38:08.575651 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0217e98-1f36-44a6-a74c-57bad28b0e00 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:08.585863 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0217e98-1f36-44a6-a74c-57bad28b0e00 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:08.590341 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3c4b2-cbac-4b12-85b9-42ac434e4484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:08.590979 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3c4b2-cbac-4b12-85b9-42ac434e4484 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.591250 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3c4b2-cbac-4b12-85b9-42ac434e4484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.591517 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3c4b2-cbac-4b12-85b9-42ac434e4484 admin admin] PATH_INFO: `/v3/projects/1392c2fb793d4f82a13c52f4d27d8f46/users/f3a05eab175e408ea67ef2839721a12c/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.606180 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3c4b2-cbac-4b12-85b9-42ac434e4484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1392c2fb793d4f82a13c52f4d27d8f46, user_id=f3a05eab175e408ea67ef2839721a12c, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:08.608758 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3c4b2-cbac-4b12-85b9-42ac434e4484 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:08.622948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1416/2816] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:38:08 2022] PUT /identity/v3/projects/1392c2fb793d4f82a13c52f4d27d8f46/users/f3a05eab175e408ea67ef2839721a12c/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:08.633228 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee3fd6-d8ac-400d-bbf2-c578a4fb4952 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.633515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee3fd6-d8ac-400d-bbf2-c578a4fb4952 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.633791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee3fd6-d8ac-400d-bbf2-c578a4fb4952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.661914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-daee3fd6-d8ac-400d-bbf2-c578a4fb4952 None None] MFA Rules not processed for user `f3a05eab175e408ea67ef2839721a12c`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:08.663152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-daee3fd6-d8ac-400d-bbf2-c578a4fb4952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:08.725575 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1401/2817] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:08.730804 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3c4b2-cbac-4b12-85b9-42ac434e4484 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:08.774720 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:08.793070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3c4b2-cbac-4b12-85b9-42ac434e4484 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:08.797882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c1a26-79ad-43b8-9517-500cf3dfc676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:08.798508 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c1a26-79ad-43b8-9517-500cf3dfc676 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.798785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c1a26-79ad-43b8-9517-500cf3dfc676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.799052 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c1a26-79ad-43b8-9517-500cf3dfc676 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.799682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c1a26-79ad-43b8-9517-500cf3dfc676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:08.801547 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c1a26-79ad-43b8-9517-500cf3dfc676 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:08.829201 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1417/2818] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:38:08 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:38:08.833219 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee3fd6-d8ac-400d-bbf2-c578a4fb4952 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:08.844612 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee3fd6-d8ac-400d-bbf2-c578a4fb4952 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:08.903780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db07f1-aecd-49f5-a8c9-1130bd2f259d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:08.904379 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db07f1-aecd-49f5-a8c9-1130bd2f259d admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.904655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db07f1-aecd-49f5-a8c9-1130bd2f259d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.904928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db07f1-aecd-49f5-a8c9-1130bd2f259d admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.905549 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db07f1-aecd-49f5-a8c9-1130bd2f259d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:08.907478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52db07f1-aecd-49f5-a8c9-1130bd2f259d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:08.928570 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1402/2819] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:38:08 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:38:08.932562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c1a26-79ad-43b8-9517-500cf3dfc676 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:08.942715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c1a26-79ad-43b8-9517-500cf3dfc676 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:08.947998 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2294a-3903-4521-9dd0-7cf9f0f13fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:08.948621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2294a-3903-4521-9dd0-7cf9f0f13fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.948897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2294a-3903-4521-9dd0-7cf9f0f13fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.949188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2294a-3903-4521-9dd0-7cf9f0f13fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.949822 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2294a-3903-4521-9dd0-7cf9f0f13fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:08.951431 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2294a-3903-4521-9dd0-7cf9f0f13fa9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:08.961997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1418/2820] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:38:08.965966 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db07f1-aecd-49f5-a8c9-1130bd2f259d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:08.975885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52db07f1-aecd-49f5-a8c9-1130bd2f259d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:08.980453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae27d3d-3d41-4b6c-b10b-bad72b7c284c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:08.981095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae27d3d-3d41-4b6c-b10b-bad72b7c284c admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:08.981377 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae27d3d-3d41-4b6c-b10b-bad72b7c284c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:08.981655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae27d3d-3d41-4b6c-b10b-bad72b7c284c admin admin] PATH_INFO: `/v3/projects/f97babaa1bb0417a91662be1f0ce54c3/users/c1566566de824365b10c5e047efcffea/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:08.996507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae27d3d-3d41-4b6c-b10b-bad72b7c284c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97babaa1bb0417a91662be1f0ce54c3, user_id=c1566566de824365b10c5e047efcffea, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:08.999198 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae27d3d-3d41-4b6c-b10b-bad72b7c284c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:09.012320 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1403/2821] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:38:08 2022] PUT /identity/v3/projects/f97babaa1bb0417a91662be1f0ce54c3/users/c1566566de824365b10c5e047efcffea/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:09.016267 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2294a-3903-4521-9dd0-7cf9f0f13fa9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:09.026770 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2294a-3903-4521-9dd0-7cf9f0f13fa9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:09.082514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd77016-9fcd-4bea-bc2d-b7b1434c0452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:09.083304 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd77016-9fcd-4bea-bc2d-b7b1434c0452 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:09.083587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd77016-9fcd-4bea-bc2d-b7b1434c0452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:09.084032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd77016-9fcd-4bea-bc2d-b7b1434c0452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:09.084753 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd77016-9fcd-4bea-bc2d-b7b1434c0452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:09.086909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd77016-9fcd-4bea-bc2d-b7b1434c0452 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:09.096633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1419/2822] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:38:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:38:09.100898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae27d3d-3d41-4b6c-b10b-bad72b7c284c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:09.111626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae27d3d-3d41-4b6c-b10b-bad72b7c284c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:09.116156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04adc7-4ac7-499f-b9ae-292839ec404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:09.116802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04adc7-4ac7-499f-b9ae-292839ec404c admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:09.117079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04adc7-4ac7-499f-b9ae-292839ec404c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:09.117378 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d04adc7-4ac7-499f-b9ae-292839ec404c admin admin] PATH_INFO: `/v3/projects/f97babaa1bb0417a91662be1f0ce54c3/users/c1566566de824365b10c5e047efcffea/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:09.119387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d04adc7-4ac7-499f-b9ae-292839ec404c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97babaa1bb0417a91662be1f0ce54c3, user_id=c1566566de824365b10c5e047efcffea, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:09.122028 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d04adc7-4ac7-499f-b9ae-292839ec404c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:09.132626 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1404/2823] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:38:09 2022] PUT /identity/v3/projects/f97babaa1bb0417a91662be1f0ce54c3/users/c1566566de824365b10c5e047efcffea/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:09.144086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534c0b9-ac4f-4ce8-860a-56b7f351756a None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:09.144375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534c0b9-ac4f-4ce8-860a-56b7f351756a None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:09.144650 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534c0b9-ac4f-4ce8-860a-56b7f351756a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:09.174197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9534c0b9-ac4f-4ce8-860a-56b7f351756a None None] MFA Rules not processed for user `c1566566de824365b10c5e047efcffea`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:09.175421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9534c0b9-ac4f-4ce8-860a-56b7f351756a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:09.251991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1420/2824] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:09.263397 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04adc7-4ac7-499f-b9ae-292839ec404c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:09.265070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7d04adc7-4ac7-499f-b9ae-292839ec404c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:09.310711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:09.323767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d04adc7-4ac7-499f-b9ae-292839ec404c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:09.327629 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376490e-e54b-4314-9adb-1eacc3e52791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:09.328220 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376490e-e54b-4314-9adb-1eacc3e52791 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:09.328493 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376490e-e54b-4314-9adb-1eacc3e52791 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:09.328761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376490e-e54b-4314-9adb-1eacc3e52791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:09.338129 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376490e-e54b-4314-9adb-1eacc3e52791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:09.339531 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376490e-e54b-4314-9adb-1eacc3e52791 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:09.348795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1405/2825] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:38:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:38:09.779593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534c0b9-ac4f-4ce8-860a-56b7f351756a None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:09.822793 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:09.845240 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534c0b9-ac4f-4ce8-860a-56b7f351756a None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:09.850265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:09.851004 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:09.851319 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:09.851589 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] PATH_INFO: `/v3/users/f3a05eab175e408ea67ef2839721a12c` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:09.853223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3a05eab175e408ea67ef2839721a12c)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:09.855139 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:09.902073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3a05eab175e408ea67ef2839721a12c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:09.909630 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3a05eab175e408ea67ef2839721a12c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:09.910278 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3a05eab175e408ea67ef2839721a12c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:09.917964 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3a05eab175e408ea67ef2839721a12c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:09.930262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3a05eab175e408ea67ef2839721a12c'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:09.936186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1421/2826] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:38:09 2022] DELETE /identity/v3/users/f3a05eab175e408ea67ef2839721a12c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:09.940784 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9f29b-5113-4caa-8a6b-4475369bbc00 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:09.941066 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9f29b-5113-4caa-8a6b-4475369bbc00 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:09.941342 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9f29b-5113-4caa-8a6b-4475369bbc00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:09.960885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bdd9f29b-5113-4caa-8a6b-4475369bbc00 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:09.962550 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bdd9f29b-5113-4caa-8a6b-4475369bbc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:10.019858 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1406/2827] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:10.030105 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:10.031528 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:10.087710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9894f7e-13dd-40da-9304-81963973cc0d admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:10.091309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab2a2e-3320-4a56-93b0-731ba631c949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:10.091911 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab2a2e-3320-4a56-93b0-731ba631c949 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:10.092190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab2a2e-3320-4a56-93b0-731ba631c949 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:10.092463 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab2a2e-3320-4a56-93b0-731ba631c949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:10.101464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab2a2e-3320-4a56-93b0-731ba631c949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:10.103518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab2a2e-3320-4a56-93b0-731ba631c949 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:10.112699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1422/2828] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:38:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:38:10.383680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9f29b-5113-4caa-8a6b-4475369bbc00 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:10.385172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bdd9f29b-5113-4caa-8a6b-4475369bbc00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:10.404962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9f29b-5113-4caa-8a6b-4475369bbc00 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:10.410106 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:10.410802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:10.411102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:10.411486 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] PATH_INFO: `/v3/projects/1392c2fb793d4f82a13c52f4d27d8f46` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:10.412705 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1392c2fb793d4f82a13c52f4d27d8f46)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:10.415910 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:10.465831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1392c2fb793d4f82a13c52f4d27d8f46', 'request_id': 'req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:10.474095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1392c2fb793d4f82a13c52f4d27d8f46', 'request_id': 'req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:10.479728 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1392c2fb793d4f82a13c52f4d27d8f46', 'request_id': 'req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:10.491339 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] The token cache is being invalidate because project 1392c2fb793d4f82a13c52f4d27d8f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:38:10.491625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:10.492260 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:10.493775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1407/2829] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:38:10 2022] DELETE /identity/v3/projects/1392c2fb793d4f82a13c52f4d27d8f46 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:10.497694 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab2a2e-3320-4a56-93b0-731ba631c949 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:10.507647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab2a2e-3320-4a56-93b0-731ba631c949 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:10.565657 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf96836-edd0-4c9f-bb31-a7edea3a346d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:10.566316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf96836-edd0-4c9f-bb31-a7edea3a346d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:10.566595 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf96836-edd0-4c9f-bb31-a7edea3a346d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:10.566866 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf96836-edd0-4c9f-bb31-a7edea3a346d admin admin] PATH_INFO: `/v3/users/c1566566de824365b10c5e047efcffea` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:10.567977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf96836-edd0-4c9f-bb31-a7edea3a346d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1566566de824365b10c5e047efcffea)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:10.569923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf96836-edd0-4c9f-bb31-a7edea3a346d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:10.573070 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:10.617281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ddf96836-edd0-4c9f-bb31-a7edea3a346d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1566566de824365b10c5e047efcffea'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:10.625507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ddf96836-edd0-4c9f-bb31-a7edea3a346d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1566566de824365b10c5e047efcffea'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:10.626281 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ddf96836-edd0-4c9f-bb31-a7edea3a346d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1566566de824365b10c5e047efcffea'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:10.633932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ddf96836-edd0-4c9f-bb31-a7edea3a346d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1566566de824365b10c5e047efcffea'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:10.653473 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-ddf96836-edd0-4c9f-bb31-a7edea3a346d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1566566de824365b10c5e047efcffea'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:10.660317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1423/2830] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:38:10 2022] DELETE /identity/v3/users/c1566566de824365b10c5e047efcffea => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:10.939679 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:10.941241 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:10.955735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40bb2c-7b4a-4d1f-b129-138cecd24d0a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:11.012970 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a6c05-74fe-47a1-887c-eae896386958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:11.013542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a6c05-74fe-47a1-887c-eae896386958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.013819 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a6c05-74fe-47a1-887c-eae896386958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.014115 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a6c05-74fe-47a1-887c-eae896386958 admin admin] PATH_INFO: `/v3/projects/f97babaa1bb0417a91662be1f0ce54c3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.015214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a6c05-74fe-47a1-887c-eae896386958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f97babaa1bb0417a91662be1f0ce54c3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:11.017255 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a6c05-74fe-47a1-887c-eae896386958 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:11.027491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:11.066675 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e42a6c05-74fe-47a1-887c-eae896386958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f97babaa1bb0417a91662be1f0ce54c3', 'request_id': 'req-e42a6c05-74fe-47a1-887c-eae896386958'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:11.074037 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e42a6c05-74fe-47a1-887c-eae896386958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f97babaa1bb0417a91662be1f0ce54c3', 'request_id': 'req-e42a6c05-74fe-47a1-887c-eae896386958'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:11.079796 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e42a6c05-74fe-47a1-887c-eae896386958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f97babaa1bb0417a91662be1f0ce54c3', 'request_id': 'req-e42a6c05-74fe-47a1-887c-eae896386958'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:11.091078 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e42a6c05-74fe-47a1-887c-eae896386958 admin admin] The token cache is being invalidate because project f97babaa1bb0417a91662be1f0ce54c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:38:11.091371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e42a6c05-74fe-47a1-887c-eae896386958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:11.092017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e42a6c05-74fe-47a1-887c-eae896386958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:11.093504 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1408/2831] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:38:10 2022] DELETE /identity/v3/projects/f97babaa1bb0417a91662be1f0ce54c3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:11.098515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933a282-3638-4e70-851c-84c0eebb7b9d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.099011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933a282-3638-4e70-851c-84c0eebb7b9d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.099136 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7933a282-3638-4e70-851c-84c0eebb7b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.119462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-7933a282-3638-4e70-851c-84c0eebb7b9d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:11.120682 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7933a282-3638-4e70-851c-84c0eebb7b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:11.179072 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1424/2832] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:11.187727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d990be-7da6-4756-8a12-d79aebd02b2a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.188017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d990be-7da6-4756-8a12-d79aebd02b2a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.188292 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d990be-7da6-4756-8a12-d79aebd02b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.210615 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d3d990be-7da6-4756-8a12-d79aebd02b2a None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:11.211871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d3d990be-7da6-4756-8a12-d79aebd02b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:11.220142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1409/2833] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:11.224198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933a282-3638-4e70-851c-84c0eebb7b9d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:11.234418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7933a282-3638-4e70-851c-84c0eebb7b9d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:11.238976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9c52a-e3ba-45c0-b284-aa6009fcd80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:11.239576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9c52a-e3ba-45c0-b284-aa6009fcd80c admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.239932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9c52a-e3ba-45c0-b284-aa6009fcd80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.240110 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9c52a-e3ba-45c0-b284-aa6009fcd80c admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.240752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9c52a-e3ba-45c0-b284-aa6009fcd80c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:38:11.241053 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9c52a-e3ba-45c0-b284-aa6009fcd80c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:11.243260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9c52a-e3ba-45c0-b284-aa6009fcd80c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:11.255073 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1425/2834] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:38:11.259361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d990be-7da6-4756-8a12-d79aebd02b2a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:11.268554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d990be-7da6-4756-8a12-d79aebd02b2a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:11.273398 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59693052-ed42-4951-ae31-a80c2f8bcf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:11.274192 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59693052-ed42-4951-ae31-a80c2f8bcf75 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.274468 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59693052-ed42-4951-ae31-a80c2f8bcf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.274737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59693052-ed42-4951-ae31-a80c2f8bcf75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.275370 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59693052-ed42-4951-ae31-a80c2f8bcf75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:11.277232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59693052-ed42-4951-ae31-a80c2f8bcf75 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:11.304787 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1410/2835] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:38:11 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:38:11.309533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9c52a-e3ba-45c0-b284-aa6009fcd80c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:11.319499 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9c52a-e3ba-45c0-b284-aa6009fcd80c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:11.374816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ee1b2-4b9e-4eeb-8f58-bd702e85dbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:11.375423 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ee1b2-4b9e-4eeb-8f58-bd702e85dbcf admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.375704 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ee1b2-4b9e-4eeb-8f58-bd702e85dbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.375977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ee1b2-4b9e-4eeb-8f58-bd702e85dbcf admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.376601 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ee1b2-4b9e-4eeb-8f58-bd702e85dbcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:11.378581 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ee1b2-4b9e-4eeb-8f58-bd702e85dbcf admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:11.394075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:11.402126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1426/2836] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:38:11 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:38:11.405821 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59693052-ed42-4951-ae31-a80c2f8bcf75 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:11.416898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59693052-ed42-4951-ae31-a80c2f8bcf75 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:11.421666 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdc911-3d21-40f5-a11a-317123385017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:11.422283 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdc911-3d21-40f5-a11a-317123385017 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.422566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdc911-3d21-40f5-a11a-317123385017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.422841 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdc911-3d21-40f5-a11a-317123385017 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.423483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcdc911-3d21-40f5-a11a-317123385017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:11.425069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcdc911-3d21-40f5-a11a-317123385017 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:11.434998 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1411/2837] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:38:11.438860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ee1b2-4b9e-4eeb-8f58-bd702e85dbcf admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:11.449144 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ee1b2-4b9e-4eeb-8f58-bd702e85dbcf admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:11.453672 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d019e1a-890c-4e2e-8655-8892ee21e105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:11.454301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d019e1a-890c-4e2e-8655-8892ee21e105 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.454576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d019e1a-890c-4e2e-8655-8892ee21e105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.454861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d019e1a-890c-4e2e-8655-8892ee21e105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.455482 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d019e1a-890c-4e2e-8655-8892ee21e105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:11.457087 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d019e1a-890c-4e2e-8655-8892ee21e105 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:11.469441 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1427/2838] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:38:11.474041 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdc911-3d21-40f5-a11a-317123385017 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:11.484907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdc911-3d21-40f5-a11a-317123385017 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:11.490690 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60362d-c7d4-4e6f-bd95-55d62d168f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:11.491351 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60362d-c7d4-4e6f-bd95-55d62d168f17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.491704 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60362d-c7d4-4e6f-bd95-55d62d168f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.491987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60362d-c7d4-4e6f-bd95-55d62d168f17 admin admin] PATH_INFO: `/v3/projects/0c63f1eab32c4624a9df34fc558cf52f/users/a17b84f7bb4c41bd86cd88e2a710ed47/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.507144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60362d-c7d4-4e6f-bd95-55d62d168f17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c63f1eab32c4624a9df34fc558cf52f, user_id=a17b84f7bb4c41bd86cd88e2a710ed47, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:11.509793 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60362d-c7d4-4e6f-bd95-55d62d168f17 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:11.522990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1412/2839] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:38:11 2022] PUT /identity/v3/projects/0c63f1eab32c4624a9df34fc558cf52f/users/a17b84f7bb4c41bd86cd88e2a710ed47/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:11.534491 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650226a6-c39a-4179-ab6b-d45c33b1e4ea None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.534803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650226a6-c39a-4179-ab6b-d45c33b1e4ea None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.535035 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650226a6-c39a-4179-ab6b-d45c33b1e4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.562501 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-650226a6-c39a-4179-ab6b-d45c33b1e4ea None None] MFA Rules not processed for user `a17b84f7bb4c41bd86cd88e2a710ed47`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:11.563735 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-650226a6-c39a-4179-ab6b-d45c33b1e4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:11.626947 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1428/2840] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:11.687408 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60362d-c7d4-4e6f-bd95-55d62d168f17 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:11.750322 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60362d-c7d4-4e6f-bd95-55d62d168f17 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:11.754948 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59a970-8894-43f3-95fc-a76d46082dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:11.755548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59a970-8894-43f3-95fc-a76d46082dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.755828 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59a970-8894-43f3-95fc-a76d46082dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.756101 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59a970-8894-43f3-95fc-a76d46082dd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.756721 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59a970-8894-43f3-95fc-a76d46082dd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:11.758700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b59a970-8894-43f3-95fc-a76d46082dd3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:11.784964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1413/2841] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:38:11 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:38:11.789141 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650226a6-c39a-4179-ab6b-d45c33b1e4ea None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:11.800533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650226a6-c39a-4179-ab6b-d45c33b1e4ea None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:11.857919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec5af5-2501-44dd-be27-f4c9a2bb14c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:11.858558 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec5af5-2501-44dd-be27-f4c9a2bb14c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.858840 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec5af5-2501-44dd-be27-f4c9a2bb14c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.859114 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec5af5-2501-44dd-be27-f4c9a2bb14c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.859743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec5af5-2501-44dd-be27-f4c9a2bb14c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:11.861655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec5af5-2501-44dd-be27-f4c9a2bb14c8 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:11.882755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1429/2842] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:38:11 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:38:11.887395 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59a970-8894-43f3-95fc-a76d46082dd3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:11.897662 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59a970-8894-43f3-95fc-a76d46082dd3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:11.903307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd812a-96c5-4798-bb3f-a7d72d37c4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:11.903907 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd812a-96c5-4798-bb3f-a7d72d37c4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.904187 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd812a-96c5-4798-bb3f-a7d72d37c4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.904461 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd812a-96c5-4798-bb3f-a7d72d37c4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.905121 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd812a-96c5-4798-bb3f-a7d72d37c4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:11.906771 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd812a-96c5-4798-bb3f-a7d72d37c4a6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:11.917433 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1414/2843] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:38:11.921564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec5af5-2501-44dd-be27-f4c9a2bb14c8 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:11.931573 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec5af5-2501-44dd-be27-f4c9a2bb14c8 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:11.936196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d160a-3ae7-4a2a-bf84-9358237bfb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:11.936874 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d160a-3ae7-4a2a-bf84-9358237bfb9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:11.937157 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d160a-3ae7-4a2a-bf84-9358237bfb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:11.937433 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d160a-3ae7-4a2a-bf84-9358237bfb9e admin admin] PATH_INFO: `/v3/projects/ba94df6e60854801a281a369e273cc9c/users/4708abad24a54263807756cf29af64ce/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:11.953438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d160a-3ae7-4a2a-bf84-9358237bfb9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba94df6e60854801a281a369e273cc9c, user_id=4708abad24a54263807756cf29af64ce, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:11.956169 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d160a-3ae7-4a2a-bf84-9358237bfb9e admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:11.969993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1430/2844] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:38:11 2022] PUT /identity/v3/projects/ba94df6e60854801a281a369e273cc9c/users/4708abad24a54263807756cf29af64ce/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:11.974162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd812a-96c5-4798-bb3f-a7d72d37c4a6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:11.984337 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd812a-96c5-4798-bb3f-a7d72d37c4a6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:11.989092 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:12.043811 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60267bc8-ee31-4179-adf6-a4b339411ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:12.044327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60267bc8-ee31-4179-adf6-a4b339411ebb admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:12.044602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60267bc8-ee31-4179-adf6-a4b339411ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:12.044871 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60267bc8-ee31-4179-adf6-a4b339411ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:12.045507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60267bc8-ee31-4179-adf6-a4b339411ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:12.047108 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60267bc8-ee31-4179-adf6-a4b339411ebb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:12.057199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1415/2845] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:38:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:38:12.061510 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d160a-3ae7-4a2a-bf84-9358237bfb9e admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:12.073121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d160a-3ae7-4a2a-bf84-9358237bfb9e admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:12.078593 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4491ee-2636-4efe-b3a0-3c533671b6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:12.079322 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4491ee-2636-4efe-b3a0-3c533671b6c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:12.079659 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4491ee-2636-4efe-b3a0-3c533671b6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:12.079918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4491ee-2636-4efe-b3a0-3c533671b6c9 admin admin] PATH_INFO: `/v3/projects/ba94df6e60854801a281a369e273cc9c/users/4708abad24a54263807756cf29af64ce/roles/459d01a218314543a7a8329e88b78331` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:12.082333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4491ee-2636-4efe-b3a0-3c533671b6c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba94df6e60854801a281a369e273cc9c, user_id=4708abad24a54263807756cf29af64ce, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:12.084982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4491ee-2636-4efe-b3a0-3c533671b6c9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:12.096232 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1431/2846] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:38:12 2022] PUT /identity/v3/projects/ba94df6e60854801a281a369e273cc9c/users/4708abad24a54263807756cf29af64ce/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:12.106072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad29b8f-4c4c-4dfb-9928-5b54436a7e51 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:12.106436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad29b8f-4c4c-4dfb-9928-5b54436a7e51 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:12.106735 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad29b8f-4c4c-4dfb-9928-5b54436a7e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:12.134776 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9ad29b8f-4c4c-4dfb-9928-5b54436a7e51 None None] MFA Rules not processed for user `4708abad24a54263807756cf29af64ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:12.135968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9ad29b8f-4c4c-4dfb-9928-5b54436a7e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:12.209013 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1416/2847] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:12 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:12.220218 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4491ee-2636-4efe-b3a0-3c533671b6c9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:12.221932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-de4491ee-2636-4efe-b3a0-3c533671b6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:12.253818 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:12.280929 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4491ee-2636-4efe-b3a0-3c533671b6c9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:12.284531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64f729-ee1e-4c04-ba70-03bd93e1dc79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:12.285129 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64f729-ee1e-4c04-ba70-03bd93e1dc79 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:12.285409 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64f729-ee1e-4c04-ba70-03bd93e1dc79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:12.285679 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64f729-ee1e-4c04-ba70-03bd93e1dc79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:12.294576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64f729-ee1e-4c04-ba70-03bd93e1dc79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:12.295934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64f729-ee1e-4c04-ba70-03bd93e1dc79 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:12.305659 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1432/2848] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:38:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:38:12.379857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad29b8f-4c4c-4dfb-9928-5b54436a7e51 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:12.390394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad29b8f-4c4c-4dfb-9928-5b54436a7e51 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:12.394854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306968e-1d96-4296-8a55-c0ffdcb1640b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:12.395501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306968e-1d96-4296-8a55-c0ffdcb1640b service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:12.395786 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306968e-1d96-4296-8a55-c0ffdcb1640b service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:12.396055 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306968e-1d96-4296-8a55-c0ffdcb1640b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:12.454238 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9306968e-1d96-4296-8a55-c0ffdcb1640b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:12.455681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9306968e-1d96-4296-8a55-c0ffdcb1640b service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:12.464891 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1417/2849] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:38:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:38:15.185191 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64f729-ee1e-4c04-ba70-03bd93e1dc79 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:15.254011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64f729-ee1e-4c04-ba70-03bd93e1dc79 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:15.259578 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:15.260235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:15.260514 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:15.260817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] PATH_INFO: `/v3/users/a17b84f7bb4c41bd86cd88e2a710ed47` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:15.262006 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a17b84f7bb4c41bd86cd88e2a710ed47)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:15.263976 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:15.313684 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a17b84f7bb4c41bd86cd88e2a710ed47'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:15.322270 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a17b84f7bb4c41bd86cd88e2a710ed47'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:15.322865 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a17b84f7bb4c41bd86cd88e2a710ed47'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:15.329816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a17b84f7bb4c41bd86cd88e2a710ed47'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:15.342366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a17b84f7bb4c41bd86cd88e2a710ed47'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:15.348326 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1433/2850] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:38:15 2022] DELETE /identity/v3/users/a17b84f7bb4c41bd86cd88e2a710ed47 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:15.353008 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c9e3c-fea4-482f-9a6d-f946b25e182d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:15.353519 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c9e3c-fea4-482f-9a6d-f946b25e182d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:15.353739 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c9e3c-fea4-482f-9a6d-f946b25e182d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:15.373362 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-607c9e3c-fea4-482f-9a6d-f946b25e182d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:15.374593 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-607c9e3c-fea4-482f-9a6d-f946b25e182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:15.398205 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:15.432437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1418/2851] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:38:15.442171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:15.443564 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:15.500386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfdd97-b06c-4ff3-bd6e-a689b5bc7f14 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:15.503919 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac467f0a-233d-448d-9b8f-559bd17572a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:15.504518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac467f0a-233d-448d-9b8f-559bd17572a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:15.504794 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac467f0a-233d-448d-9b8f-559bd17572a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:15.505064 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac467f0a-233d-448d-9b8f-559bd17572a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:15.511441 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:15.516560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac467f0a-233d-448d-9b8f-559bd17572a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:15.518345 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac467f0a-233d-448d-9b8f-559bd17572a4 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:15.528927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1434/2852] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:38:15.795475 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c9e3c-fea4-482f-9a6d-f946b25e182d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:15.797014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-607c9e3c-fea4-482f-9a6d-f946b25e182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:15.809627 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c9e3c-fea4-482f-9a6d-f946b25e182d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:15.814330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:15.814962 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:15.815243 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:15.815513 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] PATH_INFO: `/v3/projects/0c63f1eab32c4624a9df34fc558cf52f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:15.816638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c63f1eab32c4624a9df34fc558cf52f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:15.818565 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:15.870960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c63f1eab32c4624a9df34fc558cf52f', 'request_id': 'req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:15.882359 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c63f1eab32c4624a9df34fc558cf52f', 'request_id': 'req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:15.891899 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c63f1eab32c4624a9df34fc558cf52f', 'request_id': 'req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:15.903922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] The token cache is being invalidate because project 0c63f1eab32c4624a9df34fc558cf52f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:38:15.904228 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:15.904857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:15.906367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1419/2853] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:38:15 2022] DELETE /identity/v3/projects/0c63f1eab32c4624a9df34fc558cf52f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:15.910353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac467f0a-233d-448d-9b8f-559bd17572a4 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:15.920527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac467f0a-233d-448d-9b8f-559bd17572a4 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:15.982292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f8d4b-9c76-4aec-bffd-6bdf08b21d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:15.983068 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f8d4b-9c76-4aec-bffd-6bdf08b21d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:15.983177 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f8d4b-9c76-4aec-bffd-6bdf08b21d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:15.983447 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f8d4b-9c76-4aec-bffd-6bdf08b21d55 admin admin] PATH_INFO: `/v3/users/4708abad24a54263807756cf29af64ce` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:15.984611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f8d4b-9c76-4aec-bffd-6bdf08b21d55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4708abad24a54263807756cf29af64ce)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:15.986544 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f8d4b-9c76-4aec-bffd-6bdf08b21d55 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:16.034765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d79f8d4b-9c76-4aec-bffd-6bdf08b21d55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4708abad24a54263807756cf29af64ce'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:16.043235 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d79f8d4b-9c76-4aec-bffd-6bdf08b21d55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4708abad24a54263807756cf29af64ce'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:16.043965 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d79f8d4b-9c76-4aec-bffd-6bdf08b21d55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4708abad24a54263807756cf29af64ce'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:16.051574 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d79f8d4b-9c76-4aec-bffd-6bdf08b21d55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4708abad24a54263807756cf29af64ce'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:16.064187 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d79f8d4b-9c76-4aec-bffd-6bdf08b21d55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4708abad24a54263807756cf29af64ce'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:16.070388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1435/2854] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:38:15 2022] DELETE /identity/v3/users/4708abad24a54263807756cf29af64ce => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:16.345867 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:16.347422 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:16.366576 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7fc9c-b4dd-4a3e-828b-7c855ed681ea admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:16.430217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:16.430849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:16.431132 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:16.431410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb admin admin] PATH_INFO: `/v3/projects/ba94df6e60854801a281a369e273cc9c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:16.432529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba94df6e60854801a281a369e273cc9c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:16.434483 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:16.490498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba94df6e60854801a281a369e273cc9c', 'request_id': 'req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:16.494240 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:16.499067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba94df6e60854801a281a369e273cc9c', 'request_id': 'req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:16.506964 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba94df6e60854801a281a369e273cc9c', 'request_id': 'req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:16.519636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb admin admin] The token cache is being invalidate because project ba94df6e60854801a281a369e273cc9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:38:16.519945 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:16.520598 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-86ffe493-c5bb-46c4-b952-29ac30d0b5fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:38:16.522127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1420/2855] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:38:16 2022] DELETE /identity/v3/projects/ba94df6e60854801a281a369e273cc9c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:16.527623 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a4be8-c7ba-4987-bf93-3cc3696ce94d None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:16.528083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a4be8-c7ba-4987-bf93-3cc3696ce94d None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:16.528205 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a4be8-c7ba-4987-bf93-3cc3696ce94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:16.550214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-d54a4be8-c7ba-4987-bf93-3cc3696ce94d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:16.551555 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d54a4be8-c7ba-4987-bf93-3cc3696ce94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:16.609444 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1436/2856] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:16.616954 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a52d1-32a9-4acc-a74e-8f1001d41ce7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:16.617236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a52d1-32a9-4acc-a74e-8f1001d41ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:16.617507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a52d1-32a9-4acc-a74e-8f1001d41ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:16.637233 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2a5a52d1-32a9-4acc-a74e-8f1001d41ce7 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:16.638453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2a5a52d1-32a9-4acc-a74e-8f1001d41ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:16.646163 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1421/2857] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:16.650892 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a4be8-c7ba-4987-bf93-3cc3696ce94d None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:16.660359 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a4be8-c7ba-4987-bf93-3cc3696ce94d None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:16.664909 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c71842-b7c7-4b12-b562-eaf07659f572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:16.665492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c71842-b7c7-4b12-b562-eaf07659f572 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:16.665763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c71842-b7c7-4b12-b562-eaf07659f572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:16.666034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c71842-b7c7-4b12-b562-eaf07659f572 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:16.666695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c71842-b7c7-4b12-b562-eaf07659f572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:38:16.666990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c71842-b7c7-4b12-b562-eaf07659f572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:16.668539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c71842-b7c7-4b12-b562-eaf07659f572 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:16.676875 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:16.682800 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1437/2858] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:38:16.687194 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a52d1-32a9-4acc-a74e-8f1001d41ce7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:16.697617 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a52d1-32a9-4acc-a74e-8f1001d41ce7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:16.702277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f1695-56c5-4220-867b-2195f69dcfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:16.702868 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f1695-56c5-4220-867b-2195f69dcfb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:16.703147 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f1695-56c5-4220-867b-2195f69dcfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:16.703420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227f1695-56c5-4220-867b-2195f69dcfb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:16.704057 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f1695-56c5-4220-867b-2195f69dcfb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:16.705938 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227f1695-56c5-4220-867b-2195f69dcfb4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:16.733824 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1422/2859] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:38:16 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:38:16.737924 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c71842-b7c7-4b12-b562-eaf07659f572 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:16.748324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c71842-b7c7-4b12-b562-eaf07659f572 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:16.810407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ef183-2c59-4960-ba25-3cdbe72cbb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:16.811011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ef183-2c59-4960-ba25-3cdbe72cbb68 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:16.811291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ef183-2c59-4960-ba25-3cdbe72cbb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:16.811560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4ef183-2c59-4960-ba25-3cdbe72cbb68 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:16.812178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ef183-2c59-4960-ba25-3cdbe72cbb68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:16.814088 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4ef183-2c59-4960-ba25-3cdbe72cbb68 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:16.835882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1438/2860] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:38:16 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:38:16.840191 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f1695-56c5-4220-867b-2195f69dcfb4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:16.850491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227f1695-56c5-4220-867b-2195f69dcfb4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:16.854957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454980e-ac53-4c3d-907b-ef262d9f4307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:16.855546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454980e-ac53-4c3d-907b-ef262d9f4307 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:16.855820 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454980e-ac53-4c3d-907b-ef262d9f4307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:16.856090 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454980e-ac53-4c3d-907b-ef262d9f4307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:16.856720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454980e-ac53-4c3d-907b-ef262d9f4307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:16.858321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454980e-ac53-4c3d-907b-ef262d9f4307 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:16.873021 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1423/2861] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:38:16.873208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ef183-2c59-4960-ba25-3cdbe72cbb68 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:16.883154 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4ef183-2c59-4960-ba25-3cdbe72cbb68 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:16.888332 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ada5f-4ce6-41d0-a9c6-a2d5e6b2bbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:16.888933 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ada5f-4ce6-41d0-a9c6-a2d5e6b2bbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:16.889208 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ada5f-4ce6-41d0-a9c6-a2d5e6b2bbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:16.889505 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ada5f-4ce6-41d0-a9c6-a2d5e6b2bbb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:16.890171 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ada5f-4ce6-41d0-a9c6-a2d5e6b2bbb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:16.891740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ada5f-4ce6-41d0-a9c6-a2d5e6b2bbb5 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:16.902366 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1439/2862] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:38:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:38:16.906737 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454980e-ac53-4c3d-907b-ef262d9f4307 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:16.916573 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454980e-ac53-4c3d-907b-ef262d9f4307 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:16.921089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56934ca6-6df5-425d-af4c-1b844ecbe46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:16.921722 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56934ca6-6df5-425d-af4c-1b844ecbe46e admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:16.921996 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56934ca6-6df5-425d-af4c-1b844ecbe46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:16.922300 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56934ca6-6df5-425d-af4c-1b844ecbe46e admin admin] PATH_INFO: `/v3/projects/fa3848dc401942bdacf25682e40ef3ce/users/c838551b14324b14a460c99a33a8ece1/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:16.937134 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56934ca6-6df5-425d-af4c-1b844ecbe46e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa3848dc401942bdacf25682e40ef3ce, user_id=c838551b14324b14a460c99a33a8ece1, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:16.939843 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56934ca6-6df5-425d-af4c-1b844ecbe46e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:16.954394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1424/2863] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:38:16 2022] PUT /identity/v3/projects/fa3848dc401942bdacf25682e40ef3ce/users/c838551b14324b14a460c99a33a8ece1/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:38:16.965949 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63bfb2-c1c7-42e4-83f0-661240966011 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:16.966385 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63bfb2-c1c7-42e4-83f0-661240966011 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:16.966791 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63bfb2-c1c7-42e4-83f0-661240966011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:16.994348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-ba63bfb2-c1c7-42e4-83f0-661240966011 None None] MFA Rules not processed for user `c838551b14324b14a460c99a33a8ece1`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:38:16.995544 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ba63bfb2-c1c7-42e4-83f0-661240966011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:17.058787 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1440/2864] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:38:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:38:17.070184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56934ca6-6df5-425d-af4c-1b844ecbe46e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:17.074128 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-56934ca6-6df5-425d-af4c-1b844ecbe46e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:17.159354 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56934ca6-6df5-425d-af4c-1b844ecbe46e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:17.163023 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14496c12-e0e3-4c1b-b7ae-dbb24498461a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2d76fb9cbdd40f9b2a748209cee5b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:17.163625 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14496c12-e0e3-4c1b-b7ae-dbb24498461a service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:17.163921 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14496c12-e0e3-4c1b-b7ae-dbb24498461a service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:17.164216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14496c12-e0e3-4c1b-b7ae-dbb24498461a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:17.173193 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14496c12-e0e3-4c1b-b7ae-dbb24498461a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:17.174649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14496c12-e0e3-4c1b-b7ae-dbb24498461a service glance] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:17.184169 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1425/2865] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:38:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:38:27.055462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:27.179551 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=68,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:38:55.024546 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63bfb2-c1c7-42e4-83f0-661240966011 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:38:55.025948 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ba63bfb2-c1c7-42e4-83f0-661240966011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:55.084680 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63bfb2-c1c7-42e4-83f0-661240966011 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:38:55.089129 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccb825-06d1-4b09-8f3c-5e42bcd80705 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:38:55.089978 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccb825-06d1-4b09-8f3c-5e42bcd80705 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:38:55.090313 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccb825-06d1-4b09-8f3c-5e42bcd80705 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:38:55.090640 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccb825-06d1-4b09-8f3c-5e42bcd80705 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:38:55.093028 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-39ccb825-06d1-4b09-8f3c-5e42bcd80705 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:38:55.163921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ccb825-06d1-4b09-8f3c-5e42bcd80705 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:38:55.165348 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ccb825-06d1-4b09-8f3c-5e42bcd80705 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:38:55.176825 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1441/2866] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:38:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:39:05.171899 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=54 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:39:13.746235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14496c12-e0e3-4c1b-b7ae-dbb24498461a service glance] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:39:13.747763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-14496c12-e0e3-4c1b-b7ae-dbb24498461a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:39:13.803637 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14496c12-e0e3-4c1b-b7ae-dbb24498461a service glance] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:39:13.807199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fcb1c-78f8-473c-b4d0-f4ab2266a0e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:39:13.807791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fcb1c-78f8-473c-b4d0-f4ab2266a0e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:39:13.808065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fcb1c-78f8-473c-b4d0-f4ab2266a0e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:39:13.808336 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91fcb1c-78f8-473c-b4d0-f4ab2266a0e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:39:13.810032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b91fcb1c-78f8-473c-b4d0-f4ab2266a0e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:39:13.819930 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fcb1c-78f8-473c-b4d0-f4ab2266a0e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:39:13.821325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91fcb1c-78f8-473c-b4d0-f4ab2266a0e4 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:39:13.830075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1426/2867] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:39:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:39:23.825985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=26 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:41:21.006889 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccb825-06d1-4b09-8f3c-5e42bcd80705 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:41:21.008851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-39ccb825-06d1-4b09-8f3c-5e42bcd80705 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:41:21.148484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccb825-06d1-4b09-8f3c-5e42bcd80705 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:41:21.154069 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1a50f-233f-4c6a-91e6-4c528c9702a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:41:21.154676 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a50f-233f-4c6a-91e6-4c528c9702a1 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:41:21.154953 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a50f-233f-4c6a-91e6-4c528c9702a1 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:41:21.155221 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a50f-233f-4c6a-91e6-4c528c9702a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:41:21.156885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-dfe1a50f-233f-4c6a-91e6-4c528c9702a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:41:21.168212 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1a50f-233f-4c6a-91e6-4c528c9702a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:41:21.169747 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1a50f-233f-4c6a-91e6-4c528c9702a1 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:41:21.182690 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1442/2868] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:41:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:41:31.175297 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=63 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:41:33.311670 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fcb1c-78f8-473c-b4d0-f4ab2266a0e4 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:41:33.348898 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91fcb1c-78f8-473c-b4d0-f4ab2266a0e4 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:41:33.352447 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20772bed-e421-498d-af5d-d69062b8fca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:41:33.353708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20772bed-e421-498d-af5d-d69062b8fca9 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:41:33.353985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20772bed-e421-498d-af5d-d69062b8fca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:41:33.354284 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20772bed-e421-498d-af5d-d69062b8fca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:41:33.355987 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-20772bed-e421-498d-af5d-d69062b8fca9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:41:33.434390 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20772bed-e421-498d-af5d-d69062b8fca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:41:33.435545 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20772bed-e421-498d-af5d-d69062b8fca9 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:41:33.444296 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1427/2869] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:41:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:41:43.440313 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=49 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:42:05.457289 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1a50f-233f-4c6a-91e6-4c528c9702a1 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:42:05.468842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1a50f-233f-4c6a-91e6-4c528c9702a1 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:42:05.472375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b5c25-4a0d-43c4-a9a8-821222398f60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:42:05.472951 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b5c25-4a0d-43c4-a9a8-821222398f60 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:42:05.473227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b5c25-4a0d-43c4-a9a8-821222398f60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:42:05.473493 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b5c25-4a0d-43c4-a9a8-821222398f60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:42:05.475228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e83b5c25-4a0d-43c4-a9a8-821222398f60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:42:05.548194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b5c25-4a0d-43c4-a9a8-821222398f60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:42:05.549618 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b5c25-4a0d-43c4-a9a8-821222398f60 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:42:05.558539 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1443/2870] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:42:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:42:15.554407 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=33 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:43:17.586939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20772bed-e421-498d-af5d-d69062b8fca9 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:43:17.588406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-20772bed-e421-498d-af5d-d69062b8fca9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:43:17.612366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20772bed-e421-498d-af5d-d69062b8fca9 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:43:17.615972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9100a0-b5cf-49d6-ba00-a02daef6949c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:43:17.616569 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9100a0-b5cf-49d6-ba00-a02daef6949c service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:43:17.616849 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9100a0-b5cf-49d6-ba00-a02daef6949c service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:43:17.617126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9100a0-b5cf-49d6-ba00-a02daef6949c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:43:17.626411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9100a0-b5cf-49d6-ba00-a02daef6949c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:43:17.627866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9100a0-b5cf-49d6-ba00-a02daef6949c service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:43:17.636709 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1428/2871] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:43:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:43:27.632665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=11 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:44:07.208507 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b5c25-4a0d-43c4-a9a8-821222398f60 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:44:07.210674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e83b5c25-4a0d-43c4-a9a8-821222398f60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:44:07.268222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b5c25-4a0d-43c4-a9a8-821222398f60 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:44:07.274479 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9431110-b02e-4a0c-a079-08bb59b61fa9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:44:07.274882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9431110-b02e-4a0c-a079-08bb59b61fa9 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:44:07.275162 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9431110-b02e-4a0c-a079-08bb59b61fa9 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:44:07.275436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9431110-b02e-4a0c-a079-08bb59b61fa9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:44:07.277667 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c9431110-b02e-4a0c-a079-08bb59b61fa9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:44:07.291333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9431110-b02e-4a0c-a079-08bb59b61fa9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:44:07.292977 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9431110-b02e-4a0c-a079-08bb59b61fa9 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:44:07.307084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1444/2872] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:44:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:44:07.462407 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9100a0-b5cf-49d6-ba00-a02daef6949c service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:44:07.463930 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2c9100a0-b5cf-49d6-ba00-a02daef6949c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:44:07.524700 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9100a0-b5cf-49d6-ba00-a02daef6949c service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:44:07.529321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d693b8c-7639-487e-be75-e4d13a3f909b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:44:07.530273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d693b8c-7639-487e-be75-e4d13a3f909b service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:44:07.531074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d693b8c-7639-487e-be75-e4d13a3f909b service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:44:07.531451 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d693b8c-7639-487e-be75-e4d13a3f909b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:44:07.533591 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6d693b8c-7639-487e-be75-e4d13a3f909b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:44:07.553266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d693b8c-7639-487e-be75-e4d13a3f909b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:44:07.554822 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d693b8c-7639-487e-be75-e4d13a3f909b service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:44:07.570440 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1429/2873] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:44:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 16:44:08.702896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9431110-b02e-4a0c-a079-08bb59b61fa9 service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:44:08.715674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9431110-b02e-4a0c-a079-08bb59b61fa9 service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:44:08.723900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f502d5-1612-4de7-8112-56e290ac9613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:44:08.724634 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f502d5-1612-4de7-8112-56e290ac9613 service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:44:08.725110 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f502d5-1612-4de7-8112-56e290ac9613 service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:44:08.725392 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f502d5-1612-4de7-8112-56e290ac9613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:44:08.727167 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d6f502d5-1612-4de7-8112-56e290ac9613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:44:08.740057 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f502d5-1612-4de7-8112-56e290ac9613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:44:08.741668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f502d5-1612-4de7-8112-56e290ac9613 service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:44:08.753829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1445/2874] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:44:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:44:09.906411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d693b8c-7639-487e-be75-e4d13a3f909b service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:44:09.915127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d693b8c-7639-487e-be75-e4d13a3f909b service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:44:09.918708 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791e67b-45b8-4a3e-a350-39472d00765b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:44:09.919290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791e67b-45b8-4a3e-a350-39472d00765b service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:44:09.919566 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791e67b-45b8-4a3e-a350-39472d00765b service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:44:09.919834 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791e67b-45b8-4a3e-a350-39472d00765b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:44:09.930271 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791e67b-45b8-4a3e-a350-39472d00765b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:44:09.931667 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791e67b-45b8-4a3e-a350-39472d00765b service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:44:09.943079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1430/2875] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:44:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:44:18.748782 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=24 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:44:19.734477 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f502d5-1612-4de7-8112-56e290ac9613 service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:44:19.743723 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f502d5-1612-4de7-8112-56e290ac9613 service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:44:19.747335 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abbbbae-4e6e-490d-91e4-22594571587b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:44:19.747923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbbbae-4e6e-490d-91e4-22594571587b service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:44:19.748198 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbbbae-4e6e-490d-91e4-22594571587b service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:44:19.748467 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abbbbae-4e6e-490d-91e4-22594571587b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:44:19.756200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbbbae-4e6e-490d-91e4-22594571587b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:44:19.757602 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abbbbae-4e6e-490d-91e4-22594571587b service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:44:19.765466 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1446/2876] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:44:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:44:19.937560 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=21 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:44:29.761839 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=6 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:46:08.847866 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791e67b-45b8-4a3e-a350-39472d00765b service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:46:08.931216 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791e67b-45b8-4a3e-a350-39472d00765b service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:46:08.935809 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:46:08.936435 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:08.936717 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:08.936993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] PATH_INFO: `/v3/users/c838551b14324b14a460c99a33a8ece1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:08.938141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c838551b14324b14a460c99a33a8ece1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:46:08.940047 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:46:08.989148 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c838551b14324b14a460c99a33a8ece1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:46:08.996865 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c838551b14324b14a460c99a33a8ece1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:46:09.008775 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c838551b14324b14a460c99a33a8ece1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:46:09.009415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c838551b14324b14a460c99a33a8ece1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:46:09.016281 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c838551b14324b14a460c99a33a8ece1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:46:09.022488 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1431/2877] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:46:08 2022] DELETE /identity/v3/users/c838551b14324b14a460c99a33a8ece1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:46:09.027029 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce1abb-2425-4297-ae27-22f29aa043f8 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:09.027567 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce1abb-2425-4297-ae27-22f29aa043f8 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:09.027683 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce1abb-2425-4297-ae27-22f29aa043f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:09.065990 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-c1ce1abb-2425-4297-ae27-22f29aa043f8 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:46:09.069014 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c1ce1abb-2425-4297-ae27-22f29aa043f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:46:09.125484 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1447/2878] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:46:09.136032 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:46:09.137812 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:46:09.195069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d91d43-a82d-48fc-8cbe-cb8402ee0b1d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:46:09.198706 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a69d31-8b4c-42a6-9921-9007e88416b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:46:09.199306 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a69d31-8b4c-42a6-9921-9007e88416b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:09.199582 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a69d31-8b4c-42a6-9921-9007e88416b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:09.199853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a69d31-8b4c-42a6-9921-9007e88416b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:09.210084 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a69d31-8b4c-42a6-9921-9007e88416b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:46:09.211515 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a69d31-8b4c-42a6-9921-9007e88416b1 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:46:09.221131 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1432/2879] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:46:09.563525 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce1abb-2425-4297-ae27-22f29aa043f8 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:46:09.564943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c1ce1abb-2425-4297-ae27-22f29aa043f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:46:09.577637 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce1abb-2425-4297-ae27-22f29aa043f8 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:46:09.582629 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f1223-d43e-492b-abfa-c4fc8082ff09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:46:09.583261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f1223-d43e-492b-abfa-c4fc8082ff09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:09.583542 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f1223-d43e-492b-abfa-c4fc8082ff09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:09.583820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f1223-d43e-492b-abfa-c4fc8082ff09 admin admin] PATH_INFO: `/v3/projects/fa3848dc401942bdacf25682e40ef3ce` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:09.584947 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f1223-d43e-492b-abfa-c4fc8082ff09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa3848dc401942bdacf25682e40ef3ce)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:46:09.586885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92f1223-d43e-492b-abfa-c4fc8082ff09 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:46:09.633974 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d92f1223-d43e-492b-abfa-c4fc8082ff09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa3848dc401942bdacf25682e40ef3ce', 'request_id': 'req-d92f1223-d43e-492b-abfa-c4fc8082ff09'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:46:09.641750 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d92f1223-d43e-492b-abfa-c4fc8082ff09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa3848dc401942bdacf25682e40ef3ce', 'request_id': 'req-d92f1223-d43e-492b-abfa-c4fc8082ff09'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:46:09.648082 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d92f1223-d43e-492b-abfa-c4fc8082ff09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa3848dc401942bdacf25682e40ef3ce', 'request_id': 'req-d92f1223-d43e-492b-abfa-c4fc8082ff09'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:46:09.658314 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d92f1223-d43e-492b-abfa-c4fc8082ff09 admin admin] The token cache is being invalidate because project fa3848dc401942bdacf25682e40ef3ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:46:09.658599 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d92f1223-d43e-492b-abfa-c4fc8082ff09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:46:09.659234 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d92f1223-d43e-492b-abfa-c4fc8082ff09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:46:09.660724 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1448/2880] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:46:09 2022] DELETE /identity/v3/projects/fa3848dc401942bdacf25682e40ef3ce => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:46:09.666730 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cf019-776d-4c47-9dbc-89275024ceea None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:09.667017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cf019-776d-4c47-9dbc-89275024ceea None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:09.667294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24cf019-776d-4c47-9dbc-89275024ceea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:09.673106 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:46:09.718699 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f24cf019-776d-4c47-9dbc-89275024ceea None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:46:09.719903 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f24cf019-776d-4c47-9dbc-89275024ceea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:46:09.778636 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1433/2881] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:46:09.786312 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0bee6-31f4-4947-b778-f54d98ad810c None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:09.786600 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0bee6-31f4-4947-b778-f54d98ad810c None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:09.786900 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0bee6-31f4-4947-b778-f54d98ad810c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:09.806777 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0bc0bee6-31f4-4947-b778-f54d98ad810c None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:46:09.807968 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0bc0bee6-31f4-4947-b778-f54d98ad810c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:46:09.815712 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1449/2882] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:46:09.820567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cf019-776d-4c47-9dbc-89275024ceea None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:46:09.830436 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24cf019-776d-4c47-9dbc-89275024ceea None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:46:09.834912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8ba7-69b4-4f80-9f58-938fccab6952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:46:09.835501 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8ba7-69b4-4f80-9f58-938fccab6952 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:09.835780 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8ba7-69b4-4f80-9f58-938fccab6952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:09.836051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8ba7-69b4-4f80-9f58-938fccab6952 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:09.836684 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd8ba7-69b4-4f80-9f58-938fccab6952 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:46:09.836982 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd8ba7-69b4-4f80-9f58-938fccab6952 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:46:09.838585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd8ba7-69b4-4f80-9f58-938fccab6952 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:46:09.849791 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1434/2883] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:46:09.854135 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0bee6-31f4-4947-b778-f54d98ad810c None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:46:09.863200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc0bee6-31f4-4947-b778-f54d98ad810c None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:46:09.868298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49826ba1-e494-4c95-bd89-a8973a521dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:46:09.868882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49826ba1-e494-4c95-bd89-a8973a521dcf admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:09.869156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49826ba1-e494-4c95-bd89-a8973a521dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:09.869421 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49826ba1-e494-4c95-bd89-a8973a521dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:09.870068 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49826ba1-e494-4c95-bd89-a8973a521dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:46:09.871937 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49826ba1-e494-4c95-bd89-a8973a521dcf admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:46:09.911761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1450/2884] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:46:09 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:46:09.915941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8ba7-69b4-4f80-9f58-938fccab6952 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:46:09.925742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8ba7-69b4-4f80-9f58-938fccab6952 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:46:09.981906 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264688b2-4175-440b-83ac-c2e638cbd3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:46:09.982610 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264688b2-4175-440b-83ac-c2e638cbd3d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:09.982889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264688b2-4175-440b-83ac-c2e638cbd3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:09.983160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264688b2-4175-440b-83ac-c2e638cbd3d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:09.983800 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264688b2-4175-440b-83ac-c2e638cbd3d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:46:09.985688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264688b2-4175-440b-83ac-c2e638cbd3d4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:46:10.007580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1435/2885] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:46:09 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:46:10.011862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49826ba1-e494-4c95-bd89-a8973a521dcf admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:46:10.022311 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49826ba1-e494-4c95-bd89-a8973a521dcf admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:46:10.028016 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc2b4f-9138-4c32-ace9-730c2c4d7ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:46:10.028733 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc2b4f-9138-4c32-ace9-730c2c4d7ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:10.028864 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc2b4f-9138-4c32-ace9-730c2c4d7ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:10.029128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc2b4f-9138-4c32-ace9-730c2c4d7ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:10.029766 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc2b4f-9138-4c32-ace9-730c2c4d7ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:46:10.031370 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc2b4f-9138-4c32-ace9-730c2c4d7ec9 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:46:10.041830 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1451/2886] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:46:10.045979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264688b2-4175-440b-83ac-c2e638cbd3d4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:46:10.056901 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264688b2-4175-440b-83ac-c2e638cbd3d4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:46:10.061695 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7639f049-e972-4a66-8efc-3d3305542159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:46:10.062380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639f049-e972-4a66-8efc-3d3305542159 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:10.062663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639f049-e972-4a66-8efc-3d3305542159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:10.062939 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639f049-e972-4a66-8efc-3d3305542159 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:10.063580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639f049-e972-4a66-8efc-3d3305542159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:46:10.065151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7639f049-e972-4a66-8efc-3d3305542159 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:46:10.075831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1436/2887] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:46:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:46:10.080256 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc2b4f-9138-4c32-ace9-730c2c4d7ec9 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:46:10.090227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc2b4f-9138-4c32-ace9-730c2c4d7ec9 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:46:10.095032 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4af0e-8927-4c12-80e5-194586d77982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:46:10.095668 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4af0e-8927-4c12-80e5-194586d77982 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:10.095946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4af0e-8927-4c12-80e5-194586d77982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:10.096219 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4af0e-8927-4c12-80e5-194586d77982 admin admin] PATH_INFO: `/v3/projects/6005ea8ec83047f8ab25ef4205377ec9/users/51915fe515f94ac2bf044fe0847cdfe8/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:10.111752 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4af0e-8927-4c12-80e5-194586d77982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6005ea8ec83047f8ab25ef4205377ec9, user_id=51915fe515f94ac2bf044fe0847cdfe8, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:46:10.114945 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e4af0e-8927-4c12-80e5-194586d77982 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:46:10.117049 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:46:10.131530 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1452/2888] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:46:10 2022] PUT /identity/v3/projects/6005ea8ec83047f8ab25ef4205377ec9/users/51915fe515f94ac2bf044fe0847cdfe8/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:46:10.141973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba9745-7fab-4737-b0ff-4952322671a9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:10.142277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba9745-7fab-4737-b0ff-4952322671a9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:10.142547 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba9745-7fab-4737-b0ff-4952322671a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:10.170294 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-58ba9745-7fab-4737-b0ff-4952322671a9 None None] MFA Rules not processed for user `51915fe515f94ac2bf044fe0847cdfe8`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:46:10.171498 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-58ba9745-7fab-4737-b0ff-4952322671a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:46:10.204275 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:46:10.238184 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1437/2889] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:46:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:46:20.128119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:INSERT=1,keystone:SELECT=2 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:46:20.235258 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=11 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:46:25.996761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4af0e-8927-4c12-80e5-194586d77982 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:46:25.998559 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-23e4af0e-8927-4c12-80e5-194586d77982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:46:26.067555 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4af0e-8927-4c12-80e5-194586d77982 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:46:26.072943 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c2c3a-5fd0-41bc-a31a-c5f6f1e6c0ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:46:26.073556 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c2c3a-5fd0-41bc-a31a-c5f6f1e6c0ca service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:26.073832 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c2c3a-5fd0-41bc-a31a-c5f6f1e6c0ca service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:26.074128 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c2c3a-5fd0-41bc-a31a-c5f6f1e6c0ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:26.075848 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-514c2c3a-5fd0-41bc-a31a-c5f6f1e6c0ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:46:26.088126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c2c3a-5fd0-41bc-a31a-c5f6f1e6c0ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:46:26.089674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c2c3a-5fd0-41bc-a31a-c5f6f1e6c0ca service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:46:26.099847 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1453/2890] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:46:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:46:32.742912 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba9745-7fab-4737-b0ff-4952322671a9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:46:32.744311 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-58ba9745-7fab-4737-b0ff-4952322671a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:46:32.824157 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba9745-7fab-4737-b0ff-4952322671a9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:46:32.831816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0572cb3-8ef1-42fb-b8b7-5a4326d299bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:46:32.832428 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0572cb3-8ef1-42fb-b8b7-5a4326d299bc service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:32.832712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0572cb3-8ef1-42fb-b8b7-5a4326d299bc service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:32.833015 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0572cb3-8ef1-42fb-b8b7-5a4326d299bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:32.834884 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c0572cb3-8ef1-42fb-b8b7-5a4326d299bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:46:32.895789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0572cb3-8ef1-42fb-b8b7-5a4326d299bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:46:32.897611 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0572cb3-8ef1-42fb-b8b7-5a4326d299bc service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:46:32.916507 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1438/2891] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:46:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:46:36.095361 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=32 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:46:42.913932 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=60 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:46:56.179146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c2c3a-5fd0-41bc-a31a-c5f6f1e6c0ca service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:46:56.180537 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-514c2c3a-5fd0-41bc-a31a-c5f6f1e6c0ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:46:56.236354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c2c3a-5fd0-41bc-a31a-c5f6f1e6c0ca service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:46:56.240475 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f6e1c-7099-42be-8c5a-fdaa5829ff06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:46:56.241074 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f6e1c-7099-42be-8c5a-fdaa5829ff06 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:46:56.241354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f6e1c-7099-42be-8c5a-fdaa5829ff06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:46:56.241627 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f6e1c-7099-42be-8c5a-fdaa5829ff06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:46:56.250025 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f6e1c-7099-42be-8c5a-fdaa5829ff06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:46:56.251485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f6e1c-7099-42be-8c5a-fdaa5829ff06 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:46:56.260745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1454/2892] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:46:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:47:04.620014 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0572cb3-8ef1-42fb-b8b7-5a4326d299bc service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:47:04.621541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c0572cb3-8ef1-42fb-b8b7-5a4326d299bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:47:04.635630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0572cb3-8ef1-42fb-b8b7-5a4326d299bc service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:47:04.642719 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df376c-3cee-45b3-8633-fc25c20ea5b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:47:04.643814 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df376c-3cee-45b3-8633-fc25c20ea5b6 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:47:04.644089 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df376c-3cee-45b3-8633-fc25c20ea5b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:47:04.644357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df376c-3cee-45b3-8633-fc25c20ea5b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:47:04.646085 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-23df376c-3cee-45b3-8633-fc25c20ea5b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:47:04.713325 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df376c-3cee-45b3-8633-fc25c20ea5b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:47:04.714946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23df376c-3cee-45b3-8633-fc25c20ea5b6 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:47:04.733724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1439/2893] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:47:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:47:06.257316 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=26 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:47:14.731067 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=36 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:49:07.418893 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f6e1c-7099-42be-8c5a-fdaa5829ff06 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:49:07.420230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-303f6e1c-7099-42be-8c5a-fdaa5829ff06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:49:07.475670 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f6e1c-7099-42be-8c5a-fdaa5829ff06 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:49:07.479292 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e2f16-17c3-4c41-9b1b-c70bf61f9e63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:49:07.479890 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e2f16-17c3-4c41-9b1b-c70bf61f9e63 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:49:07.480200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e2f16-17c3-4c41-9b1b-c70bf61f9e63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:49:07.480469 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e2f16-17c3-4c41-9b1b-c70bf61f9e63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:49:07.482248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-2a5e2f16-17c3-4c41-9b1b-c70bf61f9e63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:49:07.495755 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e2f16-17c3-4c41-9b1b-c70bf61f9e63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:49:07.497168 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5e2f16-17c3-4c41-9b1b-c70bf61f9e63 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:49:07.508244 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1455/2894] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:49:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:49:09.470410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df376c-3cee-45b3-8633-fc25c20ea5b6 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:49:09.471885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-23df376c-3cee-45b3-8633-fc25c20ea5b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:49:09.483692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df376c-3cee-45b3-8633-fc25c20ea5b6 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:49:09.487303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623403f-d687-405b-bde5-e8de35ac5868 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:49:09.487899 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623403f-d687-405b-bde5-e8de35ac5868 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:49:09.488175 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623403f-d687-405b-bde5-e8de35ac5868 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:49:09.488444 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623403f-d687-405b-bde5-e8de35ac5868 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:49:09.490680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b623403f-d687-405b-bde5-e8de35ac5868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:49:09.503816 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623403f-d687-405b-bde5-e8de35ac5868 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:49:09.505829 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b623403f-d687-405b-bde5-e8de35ac5868 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:49:09.516854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1440/2895] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:49:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:49:17.503050 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=26 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:49:19.511825 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:51:28.958807 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e2f16-17c3-4c41-9b1b-c70bf61f9e63 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:51:28.998660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e2f16-17c3-4c41-9b1b-c70bf61f9e63 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:51:29.003536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49238c-445d-4c95-a80d-4c712ebecb2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:51:29.004133 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49238c-445d-4c95-a80d-4c712ebecb2a service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:51:29.004408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49238c-445d-4c95-a80d-4c712ebecb2a service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:51:29.004674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49238c-445d-4c95-a80d-4c712ebecb2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:51:29.017426 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49238c-445d-4c95-a80d-4c712ebecb2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:51:29.019040 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49238c-445d-4c95-a80d-4c712ebecb2a service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:51:29.030038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1456/2896] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:51:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:51:39.025333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=19 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:52:09.107696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623403f-d687-405b-bde5-e8de35ac5868 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:52:09.109065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b623403f-d687-405b-bde5-e8de35ac5868 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:52:09.158543 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623403f-d687-405b-bde5-e8de35ac5868 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:52:09.162065 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb976b-0565-4d2a-aef2-9459be5a25ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:52:09.162680 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb976b-0565-4d2a-aef2-9459be5a25ea service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:52:09.162956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb976b-0565-4d2a-aef2-9459be5a25ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:52:09.163223 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafb976b-0565-4d2a-aef2-9459be5a25ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:52:09.171844 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb976b-0565-4d2a-aef2-9459be5a25ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:52:09.173231 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafb976b-0565-4d2a-aef2-9459be5a25ea service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:52:09.182453 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1441/2897] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:52:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:52:09.208994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49238c-445d-4c95-a80d-4c712ebecb2a service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:52:09.227223 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49238c-445d-4c95-a80d-4c712ebecb2a service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:52:09.231330 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e4874-9cef-474f-aba7-8e470f69174d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:52:09.231923 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e4874-9cef-474f-aba7-8e470f69174d service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:52:09.232200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e4874-9cef-474f-aba7-8e470f69174d service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:52:09.232471 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e4874-9cef-474f-aba7-8e470f69174d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:52:09.240656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e4874-9cef-474f-aba7-8e470f69174d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:52:09.242089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e4874-9cef-474f-aba7-8e470f69174d service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:52:09.251939 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1457/2898] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:52:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:52:09.299272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb976b-0565-4d2a-aef2-9459be5a25ea service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:52:09.307639 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafb976b-0565-4d2a-aef2-9459be5a25ea service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:52:09.311281 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fb4a8-390d-4ec4-adbb-c6e0ba2c9acb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:52:09.311879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fb4a8-390d-4ec4-adbb-c6e0ba2c9acb service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:52:09.312160 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fb4a8-390d-4ec4-adbb-c6e0ba2c9acb service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:52:09.312437 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fb4a8-390d-4ec4-adbb-c6e0ba2c9acb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:52:09.320689 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fb4a8-390d-4ec4-adbb-c6e0ba2c9acb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:52:09.322118 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fb4a8-390d-4ec4-adbb-c6e0ba2c9acb service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:52:09.330668 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1442/2899] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:52:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:52:19.247346 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=11 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:52:19.326856 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=29 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:54:09.756932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e4874-9cef-474f-aba7-8e470f69174d service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:54:09.817866 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e4874-9cef-474f-aba7-8e470f69174d service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:54:09.823233 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:54:09.823858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:09.824138 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:09.824413 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] PATH_INFO: `/v3/users/51915fe515f94ac2bf044fe0847cdfe8` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:09.825520 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51915fe515f94ac2bf044fe0847cdfe8)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:54:09.827440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:54:09.873675 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51915fe515f94ac2bf044fe0847cdfe8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:54:09.881397 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51915fe515f94ac2bf044fe0847cdfe8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:54:09.882067 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51915fe515f94ac2bf044fe0847cdfe8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:54:09.889862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51915fe515f94ac2bf044fe0847cdfe8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:54:09.902820 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51915fe515f94ac2bf044fe0847cdfe8'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:54:09.909143 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1458/2900] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:54:09 2022] DELETE /identity/v3/users/51915fe515f94ac2bf044fe0847cdfe8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:54:09.913464 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6ca5f-14db-4d54-ac69-291d67185117 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:09.913983 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6ca5f-14db-4d54-ac69-291d67185117 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:09.914127 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6ca5f-14db-4d54-ac69-291d67185117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:09.937113 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-88f6ca5f-14db-4d54-ac69-291d67185117 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:54:09.938377 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-88f6ca5f-14db-4d54-ac69-291d67185117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:54:10.004861 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1443/2901] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 21 16:54:10.017194 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:54:10.018817 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:54:10.098515 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfb323-4916-4bd5-a131-4a984ebbd5d2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:54:10.102118 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344aeeec-50ca-48a7-8e8f-a671df67ab5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:54:10.103206 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344aeeec-50ca-48a7-8e8f-a671df67ab5b service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:10.103480 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344aeeec-50ca-48a7-8e8f-a671df67ab5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:10.103743 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344aeeec-50ca-48a7-8e8f-a671df67ab5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:10.112416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344aeeec-50ca-48a7-8e8f-a671df67ab5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:54:10.113853 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344aeeec-50ca-48a7-8e8f-a671df67ab5b service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:54:10.123803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1459/2902] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:54:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:54:10.387798 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6ca5f-14db-4d54-ac69-291d67185117 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:54:10.389687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-88f6ca5f-14db-4d54-ac69-291d67185117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:54:10.404414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6ca5f-14db-4d54-ac69-291d67185117 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:54:10.409026 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dadf29e-17dc-431d-a431-cb0acc3c73f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:54:10.409651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadf29e-17dc-431d-a431-cb0acc3c73f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:10.409985 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadf29e-17dc-431d-a431-cb0acc3c73f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:10.410245 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dadf29e-17dc-431d-a431-cb0acc3c73f1 admin admin] PATH_INFO: `/v3/projects/6005ea8ec83047f8ab25ef4205377ec9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:10.415548 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadf29e-17dc-431d-a431-cb0acc3c73f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6005ea8ec83047f8ab25ef4205377ec9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:54:10.417557 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dadf29e-17dc-431d-a431-cb0acc3c73f1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:54:10.465019 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2dadf29e-17dc-431d-a431-cb0acc3c73f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6005ea8ec83047f8ab25ef4205377ec9', 'request_id': 'req-2dadf29e-17dc-431d-a431-cb0acc3c73f1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:54:10.472649 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2dadf29e-17dc-431d-a431-cb0acc3c73f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6005ea8ec83047f8ab25ef4205377ec9', 'request_id': 'req-2dadf29e-17dc-431d-a431-cb0acc3c73f1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:54:10.484620 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2dadf29e-17dc-431d-a431-cb0acc3c73f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6005ea8ec83047f8ab25ef4205377ec9', 'request_id': 'req-2dadf29e-17dc-431d-a431-cb0acc3c73f1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:54:10.496711 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2dadf29e-17dc-431d-a431-cb0acc3c73f1 admin admin] The token cache is being invalidate because project 6005ea8ec83047f8ab25ef4205377ec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:54:10.496999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2dadf29e-17dc-431d-a431-cb0acc3c73f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:54:10.497652 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2dadf29e-17dc-431d-a431-cb0acc3c73f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:54:10.499331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1444/2903] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:54:10 2022] DELETE /identity/v3/projects/6005ea8ec83047f8ab25ef4205377ec9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:54:10.505408 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7792622-b8c6-458e-9cf3-91177ec648e6 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:10.505744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7792622-b8c6-458e-9cf3-91177ec648e6 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:10.506121 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7792622-b8c6-458e-9cf3-91177ec648e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:10.513912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:54:10.527946 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-b7792622-b8c6-458e-9cf3-91177ec648e6 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:54:10.529147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b7792622-b8c6-458e-9cf3-91177ec648e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:54:10.585715 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1460/2904] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:54:10.595394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7457fd-d8d4-42a0-8e4e-a3b79f712bf3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:10.595676 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7457fd-d8d4-42a0-8e4e-a3b79f712bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:10.595949 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7457fd-d8d4-42a0-8e4e-a3b79f712bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:10.616248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4c7457fd-d8d4-42a0-8e4e-a3b79f712bf3 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:54:10.617432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4c7457fd-d8d4-42a0-8e4e-a3b79f712bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:54:10.629049 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1445/2905] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:54:10.631041 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7792622-b8c6-458e-9cf3-91177ec648e6 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:54:10.640470 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7792622-b8c6-458e-9cf3-91177ec648e6 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:54:10.647885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587ea32-bdaf-4939-be42-a5a5c5549997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:54:10.648486 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587ea32-bdaf-4939-be42-a5a5c5549997 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:10.648765 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587ea32-bdaf-4939-be42-a5a5c5549997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:10.649038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3587ea32-bdaf-4939-be42-a5a5c5549997 admin admin] PATH_INFO: `/v3/domains` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:10.650265 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587ea32-bdaf-4939-be42-a5a5c5549997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 16:54:10.650576 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587ea32-bdaf-4939-be42-a5a5c5549997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:54:10.652151 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3587ea32-bdaf-4939-be42-a5a5c5549997 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:54:10.663681 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1461/2906] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 16:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 16:54:10.668491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7457fd-d8d4-42a0-8e4e-a3b79f712bf3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:54:10.678264 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7457fd-d8d4-42a0-8e4e-a3b79f712bf3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:54:10.685095 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81a5a6-8d8d-4533-8306-519b59c9ca36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:54:10.685694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81a5a6-8d8d-4533-8306-519b59c9ca36 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:10.685968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81a5a6-8d8d-4533-8306-519b59c9ca36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:10.686280 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81a5a6-8d8d-4533-8306-519b59c9ca36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:10.686944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81a5a6-8d8d-4533-8306-519b59c9ca36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:54:10.688854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81a5a6-8d8d-4533-8306-519b59c9ca36 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:54:10.717974 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1446/2907] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 16:54:10 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 16:54:10.722150 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587ea32-bdaf-4939-be42-a5a5c5549997 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:54:10.732658 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3587ea32-bdaf-4939-be42-a5a5c5549997 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:54:10.787220 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fabdf-30ed-436f-b1d8-2dee2374ed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:54:10.787860 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fabdf-30ed-436f-b1d8-2dee2374ed16 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:10.788015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fabdf-30ed-436f-b1d8-2dee2374ed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:10.788286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84fabdf-30ed-436f-b1d8-2dee2374ed16 admin admin] PATH_INFO: `/v3/users` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:10.788901 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84fabdf-30ed-436f-b1d8-2dee2374ed16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:54:10.790858 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84fabdf-30ed-436f-b1d8-2dee2374ed16 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:54:10.811230 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1462/2908] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 16:54:10 2022] POST /identity/v3/users => generated 295 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 16:54:10.815365 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81a5a6-8d8d-4533-8306-519b59c9ca36 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:54:10.826630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81a5a6-8d8d-4533-8306-519b59c9ca36 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:54:10.831767 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06500896-fd5d-43b7-8b97-062075273fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:54:10.832482 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06500896-fd5d-43b7-8b97-062075273fca admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:10.833230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06500896-fd5d-43b7-8b97-062075273fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:10.833230 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06500896-fd5d-43b7-8b97-062075273fca admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:10.833763 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06500896-fd5d-43b7-8b97-062075273fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:54:10.835638 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06500896-fd5d-43b7-8b97-062075273fca admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:54:10.846352 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1447/2909] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:54:10.850487 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fabdf-30ed-436f-b1d8-2dee2374ed16 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:54:10.860324 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84fabdf-30ed-436f-b1d8-2dee2374ed16 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:54:10.864861 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4be9b-5b27-4245-8562-5351e8a1d17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:54:10.865474 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4be9b-5b27-4245-8562-5351e8a1d17d admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:10.865739 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4be9b-5b27-4245-8562-5351e8a1d17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:10.866013 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4be9b-5b27-4245-8562-5351e8a1d17d admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:10.866687 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4be9b-5b27-4245-8562-5351e8a1d17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:54:10.868261 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4be9b-5b27-4245-8562-5351e8a1d17d admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:54:10.878531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1463/2910] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 16:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 16:54:10.882348 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06500896-fd5d-43b7-8b97-062075273fca admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:54:10.892655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06500896-fd5d-43b7-8b97-062075273fca admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:54:10.897252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775d479-bdce-4d1a-b2f6-f2e298187079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:54:10.897889 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775d479-bdce-4d1a-b2f6-f2e298187079 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:10.898202 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775d479-bdce-4d1a-b2f6-f2e298187079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:10.898479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e775d479-bdce-4d1a-b2f6-f2e298187079 admin admin] PATH_INFO: `/v3/projects/58d4ad578cb94217960568e8ebc533ae/users/a8c1212529554794a9ea3be51825d871/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:10.913333 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775d479-bdce-4d1a-b2f6-f2e298187079 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d4ad578cb94217960568e8ebc533ae, user_id=a8c1212529554794a9ea3be51825d871, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:54:10.916046 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e775d479-bdce-4d1a-b2f6-f2e298187079 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:54:10.928870 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1448/2911] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 16:54:10 2022] PUT /identity/v3/projects/58d4ad578cb94217960568e8ebc533ae/users/a8c1212529554794a9ea3be51825d871/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:54:10.939123 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c891f-ff00-4848-bef7-30f0d3ff4cbb None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:10.939404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c891f-ff00-4848-bef7-30f0d3ff4cbb None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:10.939674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c891f-ff00-4848-bef7-30f0d3ff4cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:10.968633 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-0a3c891f-ff00-4848-bef7-30f0d3ff4cbb None None] MFA Rules not processed for user `a8c1212529554794a9ea3be51825d871`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 16:54:10.969928 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0a3c891f-ff00-4848-bef7-30f0d3ff4cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:54:11.032845 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:54:11.039169 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1464/2912] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 16:54:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 16:54:11.049578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775d479-bdce-4d1a-b2f6-f2e298187079 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:54:11.051302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e775d479-bdce-4d1a-b2f6-f2e298187079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:54:11.073993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:54:11.122232 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e775d479-bdce-4d1a-b2f6-f2e298187079 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:54:11.125810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9086c4-a99c-4852-82bc-12e21eee9fec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:54:11.126441 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9086c4-a99c-4852-82bc-12e21eee9fec service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:54:11.126720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9086c4-a99c-4852-82bc-12e21eee9fec service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:54:11.126993 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9086c4-a99c-4852-82bc-12e21eee9fec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:54:11.135720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9086c4-a99c-4852-82bc-12e21eee9fec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:54:11.137133 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9086c4-a99c-4852-82bc-12e21eee9fec service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:54:11.145694 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1449/2913] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:54:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:54:21.040506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:54:21.141829 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=22 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:56:09.759156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c891f-ff00-4848-bef7-30f0d3ff4cbb None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:56:09.760598 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-0a3c891f-ff00-4848-bef7-30f0d3ff4cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:56:09.817639 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c891f-ff00-4848-bef7-30f0d3ff4cbb None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:56:09.821086 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a16bf3-3367-4a32-a7cd-8345365d8ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:56:09.821688 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a16bf3-3367-4a32-a7cd-8345365d8ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:56:09.821969 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a16bf3-3367-4a32-a7cd-8345365d8ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:56:09.822272 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a16bf3-3367-4a32-a7cd-8345365d8ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:56:09.824001 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d5a16bf3-3367-4a32-a7cd-8345365d8ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:56:09.906416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a16bf3-3367-4a32-a7cd-8345365d8ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:56:09.907829 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a16bf3-3367-4a32-a7cd-8345365d8ab2 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:56:09.926327 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1465/2914] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:56:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:56:13.225563 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9086c4-a99c-4852-82bc-12e21eee9fec service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:56:13.226992 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2f9086c4-a99c-4852-82bc-12e21eee9fec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:56:13.319432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9086c4-a99c-4852-82bc-12e21eee9fec service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:56:13.324428 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2809d-83b3-42e5-aa95-a55cfec08999 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:56:13.325027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2809d-83b3-42e5-aa95-a55cfec08999 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:56:13.325303 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2809d-83b3-42e5-aa95-a55cfec08999 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:56:13.325572 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2809d-83b3-42e5-aa95-a55cfec08999 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:56:13.327273 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-edf2809d-83b3-42e5-aa95-a55cfec08999 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:56:13.340424 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2809d-83b3-42e5-aa95-a55cfec08999 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:56:13.346094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf2809d-83b3-42e5-aa95-a55cfec08999 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:56:13.359141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1450/2915] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:56:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:56:16.233081 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a16bf3-3367-4a32-a7cd-8345365d8ab2 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:56:16.234792 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-d5a16bf3-3367-4a32-a7cd-8345365d8ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:56:16.248386 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a16bf3-3367-4a32-a7cd-8345365d8ab2 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:56:16.253020 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:56:16.253653 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:56:16.253932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:56:16.254309 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] PATH_INFO: `/v3/users/f619261c21204ea3b5ba624177eb8747` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:56:16.268259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f619261c21204ea3b5ba624177eb8747)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:56:16.270200 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:56:16.313590 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f619261c21204ea3b5ba624177eb8747'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.320726 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f619261c21204ea3b5ba624177eb8747'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.321352 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f619261c21204ea3b5ba624177eb8747'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.329357 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f619261c21204ea3b5ba624177eb8747'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.331255 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:56:16.344384 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f619261c21204ea3b5ba624177eb8747'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.350992 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1466/2916] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:56:16 2022] DELETE /identity/v3/users/f619261c21204ea3b5ba624177eb8747 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:56:16.523051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2809d-83b3-42e5-aa95-a55cfec08999 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:56:16.533751 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2809d-83b3-42e5-aa95-a55cfec08999 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:56:16.590795 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:56:16.591420 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:56:16.591698 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:56:16.591973 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] PATH_INFO: `/v3/projects/d529e1f3de6944f881c7fc572b1dc78e` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:56:16.593075 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d529e1f3de6944f881c7fc572b1dc78e)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:56:16.595010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:56:16.640660 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd529e1f3de6944f881c7fc572b1dc78e', 'request_id': 'req-04de537a-000d-4167-bba3-368d6b6b4989'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.647530 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd529e1f3de6944f881c7fc572b1dc78e', 'request_id': 'req-04de537a-000d-4167-bba3-368d6b6b4989'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.653176 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd529e1f3de6944f881c7fc572b1dc78e', 'request_id': 'req-04de537a-000d-4167-bba3-368d6b6b4989'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.663862 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] The token cache is being invalidate because project d529e1f3de6944f881c7fc572b1dc78e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:56:16.664147 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.664754 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.666235 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1451/2917] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:56:16 2022] DELETE /identity/v3/projects/d529e1f3de6944f881c7fc572b1dc78e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:56:16.670744 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:56:16.672178 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:56:16.685655 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1698bdb-e9b7-4878-a28c-be432a9b3ce6 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:56:16.755048 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:56:16.755819 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:56:16.756153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:56:16.756527 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] PATH_INFO: `/v3/users/3bb9ac809b2e43db922954607ff738bb` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:56:16.771381 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bb9ac809b2e43db922954607ff738bb)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:56:16.773291 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:56:16.819671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bb9ac809b2e43db922954607ff738bb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.826587 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bb9ac809b2e43db922954607ff738bb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.827259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bb9ac809b2e43db922954607ff738bb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.834078 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bb9ac809b2e43db922954607ff738bb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.846011 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bb9ac809b2e43db922954607ff738bb'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:16.851745 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1467/2918] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 16:56:16 2022] DELETE /identity/v3/users/3bb9ac809b2e43db922954607ff738bb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:56:17.028083 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:56:17.029605 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:56:17.052361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de537a-000d-4167-bba3-368d6b6b4989 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:56:17.058277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:56:17.111321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:56:17.111941 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:56:17.112215 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:56:17.112491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] PATH_INFO: `/v3/projects/70ec56eccacb4e278f80206e651d397c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:56:17.122923 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70ec56eccacb4e278f80206e651d397c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:56:17.124789 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:56:17.168981 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70ec56eccacb4e278f80206e651d397c', 'request_id': 'req-dad5fdf2-56c6-4583-9562-efa9aebf3944'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:17.175553 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70ec56eccacb4e278f80206e651d397c', 'request_id': 'req-dad5fdf2-56c6-4583-9562-efa9aebf3944'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:17.181052 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70ec56eccacb4e278f80206e651d397c', 'request_id': 'req-dad5fdf2-56c6-4583-9562-efa9aebf3944'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:17.194960 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] The token cache is being invalidate because project 70ec56eccacb4e278f80206e651d397c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 16:56:17.195401 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:17.196044 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 16:56:17.197528 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1452/2919] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 16:56:17 2022] DELETE /identity/v3/projects/70ec56eccacb4e278f80206e651d397c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 16:56:26.848430 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=64,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:56:27.191479 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:56:35.968116 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:56:35.969521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:56:36.037037 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcef11-a21d-497a-807a-bb66ad76eccb admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:56:36.095438 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cebb72-6af8-4c80-a6e5-8c1c2bd17202 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:56:36.096047 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cebb72-6af8-4c80-a6e5-8c1c2bd17202 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:56:36.096331 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cebb72-6af8-4c80-a6e5-8c1c2bd17202 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:56:36.096609 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cebb72-6af8-4c80-a6e5-8c1c2bd17202 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:56:36.098404 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-43cebb72-6af8-4c80-a6e5-8c1c2bd17202 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:56:36.110158 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cebb72-6af8-4c80-a6e5-8c1c2bd17202 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:56:36.111722 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cebb72-6af8-4c80-a6e5-8c1c2bd17202 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:56:36.122126 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1468/2920] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:56:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:56:46.117236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=58 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:57:09.415724 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:57:09.417142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:57:09.477141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5fdf2-56c6-4583-9562-efa9aebf3944 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:57:09.481681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f332-c3ae-404b-942a-4b17621765c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:57:09.482331 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f332-c3ae-404b-942a-4b17621765c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:57:09.482606 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f332-c3ae-404b-942a-4b17621765c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:57:09.482875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f332-c3ae-404b-942a-4b17621765c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:57:09.484607 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-edc5f332-c3ae-404b-942a-4b17621765c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:57:09.564630 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5f332-c3ae-404b-942a-4b17621765c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:57:09.566077 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5f332-c3ae-404b-942a-4b17621765c9 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:57:09.583946 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1453/2921] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:57:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:57:19.581298 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=61 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 16:59:10.464557 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cebb72-6af8-4c80-a6e5-8c1c2bd17202 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 16:59:10.473797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cebb72-6af8-4c80-a6e5-8c1c2bd17202 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 16:59:10.478075 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6794e07-f65b-4388-9900-52eb9c4147f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 16:59:10.478670 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6794e07-f65b-4388-9900-52eb9c4147f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 16:59:10.478944 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6794e07-f65b-4388-9900-52eb9c4147f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 16:59:10.479212 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6794e07-f65b-4388-9900-52eb9c4147f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 16:59:10.481462 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c6794e07-f65b-4388-9900-52eb9c4147f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 16:59:10.545059 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6794e07-f65b-4388-9900-52eb9c4147f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 16:59:10.546529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6794e07-f65b-4388-9900-52eb9c4147f7 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 16:59:10.555236 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1469/2922] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 16:59:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 16:59:20.551216 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=28 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:00:11.706278 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f332-c3ae-404b-942a-4b17621765c9 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:00:11.707729 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-edc5f332-c3ae-404b-942a-4b17621765c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:00:11.773443 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f332-c3ae-404b-942a-4b17621765c9 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:00:11.777914 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:00:11.778583 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:00:11.778869 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:00:11.779141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] PATH_INFO: `/v3/users/a8c1212529554794a9ea3be51825d871` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:00:11.780254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8c1212529554794a9ea3be51825d871)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:00:11.782186 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:00:11.832756 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8c1212529554794a9ea3be51825d871'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:00:11.840318 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8c1212529554794a9ea3be51825d871'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:00:11.852529 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8c1212529554794a9ea3be51825d871'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:00:11.853159 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8c1212529554794a9ea3be51825d871'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:00:11.860024 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8c1212529554794a9ea3be51825d871'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:00:11.865746 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1454/2923] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 17:00:11 2022] DELETE /identity/v3/users/a8c1212529554794a9ea3be51825d871 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 17:00:11.870877 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a202d5-8b5c-44ae-a0e1-3bbeaaf37e79 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:00:11.871330 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a202d5-8b5c-44ae-a0e1-3bbeaaf37e79 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:00:11.871428 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a202d5-8b5c-44ae-a0e1-3bbeaaf37e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:00:11.904083 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-f9a202d5-8b5c-44ae-a0e1-3bbeaaf37e79 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 17:00:11.905713 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f9a202d5-8b5c-44ae-a0e1-3bbeaaf37e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:00:11.971189 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1470/2924] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 17:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 17:00:11.982469 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:00:11.984041 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:00:12.040533 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df72dda-35de-4eaf-93df-5e2f32449c6e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:00:12.044590 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33611c61-74df-49e4-841e-bff836aa5cb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:00:12.045179 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33611c61-74df-49e4-841e-bff836aa5cb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:00:12.045456 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33611c61-74df-49e4-841e-bff836aa5cb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:00:12.045726 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33611c61-74df-49e4-841e-bff836aa5cb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:00:12.054917 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33611c61-74df-49e4-841e-bff836aa5cb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:00:12.056340 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33611c61-74df-49e4-841e-bff836aa5cb2 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:00:12.066069 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1455/2925] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:00:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:00:12.333237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a202d5-8b5c-44ae-a0e1-3bbeaaf37e79 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:00:12.335055 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-f9a202d5-8b5c-44ae-a0e1-3bbeaaf37e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:00:12.348165 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a202d5-8b5c-44ae-a0e1-3bbeaaf37e79 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:00:12.354710 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:00:12.355340 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:00:12.355615 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:00:12.355886 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] PATH_INFO: `/v3/projects/58d4ad578cb94217960568e8ebc533ae` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:00:12.357007 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d4ad578cb94217960568e8ebc533ae)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:00:12.360022 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:00:12.407981 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d4ad578cb94217960568e8ebc533ae', 'request_id': 'req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:00:12.414813 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d4ad578cb94217960568e8ebc533ae', 'request_id': 'req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:00:12.421665 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d4ad578cb94217960568e8ebc533ae', 'request_id': 'req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:00:12.431952 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] The token cache is being invalidate because project 58d4ad578cb94217960568e8ebc533ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 17:00:12.432237 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:00:12.432862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:00:12.434368 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1471/2926] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 17:00:12 2022] DELETE /identity/v3/projects/58d4ad578cb94217960568e8ebc533ae => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 17:00:22.061979 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:00:22.429018 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:01:39.937648 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33611c61-74df-49e4-841e-bff836aa5cb2 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:01:39.939051 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-33611c61-74df-49e4-841e-bff836aa5cb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:01:40.036109 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33611c61-74df-49e4-841e-bff836aa5cb2 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:01:40.093671 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381aef26-1b07-4bd4-8e36-303bd64e040c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:01:40.094296 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aef26-1b07-4bd4-8e36-303bd64e040c service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:01:40.094578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aef26-1b07-4bd4-8e36-303bd64e040c service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:01:40.094850 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aef26-1b07-4bd4-8e36-303bd64e040c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:01:40.096567 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-381aef26-1b07-4bd4-8e36-303bd64e040c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:01:40.108147 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381aef26-1b07-4bd4-8e36-303bd64e040c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:01:40.109647 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381aef26-1b07-4bd4-8e36-303bd64e040c service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:01:40.120027 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1456/2927] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:01:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:01:50.115269 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=71 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:02:10.156147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:02:10.157562 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:02:10.249560 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a54a7-1c60-4a81-bdfc-229a93f2d1bd admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:02:10.253202 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30861efa-4d1b-41ee-b9d9-4c66373e1676 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9fb56e56c7f49ce959d0bc6c515044c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:02:10.253797 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30861efa-4d1b-41ee-b9d9-4c66373e1676 service cinder] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:02:10.254101 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30861efa-4d1b-41ee-b9d9-4c66373e1676 service cinder] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:02:10.254375 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30861efa-4d1b-41ee-b9d9-4c66373e1676 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:02:10.256034 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-30861efa-4d1b-41ee-b9d9-4c66373e1676 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:02:10.312963 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30861efa-4d1b-41ee-b9d9-4c66373e1676 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:02:10.314420 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30861efa-4d1b-41ee-b9d9-4c66373e1676 service cinder] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:02:10.323068 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1472/2928] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:02:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:02:20.319119 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=65 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:03:02.802585 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea3aad-c485-4cca-8b9e-60a9a46a9a22 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:03:02.803106 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea3aad-c485-4cca-8b9e-60a9a46a9a22 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:03:02.803319 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea3aad-c485-4cca-8b9e-60a9a46a9a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:03:02.833480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a9ea3aad-c485-4cca-8b9e-60a9a46a9a22 None None] MFA Rules not processed for user `10f2730695454358bc1707dd0815d880`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 17:03:02.834720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a9ea3aad-c485-4cca-8b9e-60a9a46a9a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:02.842277 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1457/2929] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 17:03:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 17:03:02.893989 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30861efa-4d1b-41ee-b9d9-4c66373e1676 service cinder] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:03:02.895401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-30861efa-4d1b-41ee-b9d9-4c66373e1676 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:02.980577 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30861efa-4d1b-41ee-b9d9-4c66373e1676 service cinder] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:03:02.985317 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44504c7a-43f7-4555-9006-b135c470ecf6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2d76fb9cbdd40f9b2a748209cee5b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:03:02.985927 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504c7a-43f7-4555-9006-b135c470ecf6 service glance] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:03:02.986240 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504c7a-43f7-4555-9006-b135c470ecf6 service glance] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:03:02.986513 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44504c7a-43f7-4555-9006-b135c470ecf6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:03:02.988245 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-44504c7a-43f7-4555-9006-b135c470ecf6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:02.990254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea3aad-c485-4cca-8b9e-60a9a46a9a22 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:03:02.991741 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a9ea3aad-c485-4cca-8b9e-60a9a46a9a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:03.004720 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea3aad-c485-4cca-8b9e-60a9a46a9a22 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:03:03.008521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82452a-6ed2-4b4c-abc3-1d2c2e604a5e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2d76fb9cbdd40f9b2a748209cee5b75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:03:03.009124 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82452a-6ed2-4b4c-abc3-1d2c2e604a5e service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:03:03.009406 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82452a-6ed2-4b4c-abc3-1d2c2e604a5e service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:03:03.009696 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82452a-6ed2-4b4c-abc3-1d2c2e604a5e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:03:03.011562 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1a82452a-6ed2-4b4c-abc3-1d2c2e604a5e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1a82452a-6ed2-4b4c-abc3-1d2c2e604a5e service glance] Could not find project: fa3848dc401942bdacf25682e40ef3ce.: keystone.exception.ProjectNotFound: Could not find project: fa3848dc401942bdacf25682e40ef3ce. Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 21 17:03:03.043399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 17:03:03.044635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 21 17:03:03.045830 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 21 17:03:03.046927 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 21 17:03:03.048010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: fa3848dc401942bdacf25682e40ef3ce. Sep 21 17:03:03.048010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Sep 21 17:03:03.048010 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1458/2930] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:03:02 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Sep 21 17:03:03.091015 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44504c7a-43f7-4555-9006-b135c470ecf6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:03:03.092518 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44504c7a-43f7-4555-9006-b135c470ecf6 service glance] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:03:03.112938 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1473/2931] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:03:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:03:03.312368 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82452a-6ed2-4b4c-abc3-1d2c2e604a5e service glance] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:03:03.313875 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1a82452a-6ed2-4b4c-abc3-1d2c2e604a5e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:03.373682 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82452a-6ed2-4b4c-abc3-1d2c2e604a5e service glance] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:03:03.377188 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3f654-1e05-44c6-8fd0-355bb0423e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:03:03.377781 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3f654-1e05-44c6-8fd0-355bb0423e9f service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:03:03.378072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3f654-1e05-44c6-8fd0-355bb0423e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:03:03.378361 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3f654-1e05-44c6-8fd0-355bb0423e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:03:03.380070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4cd3f654-1e05-44c6-8fd0-355bb0423e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:03.389835 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3f654-1e05-44c6-8fd0-355bb0423e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:03:03.391508 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3f654-1e05-44c6-8fd0-355bb0423e9f service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:03:03.399503 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1459/2932] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:03:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:03:03.746145 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44504c7a-43f7-4555-9006-b135c470ecf6 service glance] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:03:03.747531 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-44504c7a-43f7-4555-9006-b135c470ecf6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:03.758761 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44504c7a-43f7-4555-9006-b135c470ecf6 service glance] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:03:03.762851 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8cd54-d6b9-4718-a35a-beeb65569169 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:03:03.763440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8cd54-d6b9-4718-a35a-beeb65569169 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:03:03.763721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8cd54-d6b9-4718-a35a-beeb65569169 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:03:03.763994 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8cd54-d6b9-4718-a35a-beeb65569169 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:03:03.765660 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-ecb8cd54-d6b9-4718-a35a-beeb65569169 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:03.778485 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8cd54-d6b9-4718-a35a-beeb65569169 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:03:03.779920 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8cd54-d6b9-4718-a35a-beeb65569169 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:03:03.790610 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1474/2933] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:03:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:03:04.739206 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3f654-1e05-44c6-8fd0-355bb0423e9f service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:03:04.740553 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4cd3f654-1e05-44c6-8fd0-355bb0423e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:04.755727 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3f654-1e05-44c6-8fd0-355bb0423e9f service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:03:04.760705 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31c807-a4d5-462b-8f08-8dc1179ba4e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:03:04.761295 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c807-a4d5-462b-8f08-8dc1179ba4e6 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:03:04.761579 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c807-a4d5-462b-8f08-8dc1179ba4e6 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:03:04.761855 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c807-a4d5-462b-8f08-8dc1179ba4e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:03:04.763546 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9b31c807-a4d5-462b-8f08-8dc1179ba4e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:04.774599 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31c807-a4d5-462b-8f08-8dc1179ba4e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:03:04.776164 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b31c807-a4d5-462b-8f08-8dc1179ba4e6 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:03:04.785933 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1460/2934] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:03:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:03:09.420148 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8cd54-d6b9-4718-a35a-beeb65569169 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:03:09.428812 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8cd54-d6b9-4718-a35a-beeb65569169 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:03:09.432418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b421fc-ca06-40d2-bef1-d598d4ff5ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:03:09.433031 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b421fc-ca06-40d2-bef1-d598d4ff5ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:03:09.433304 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b421fc-ca06-40d2-bef1-d598d4ff5ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:03:09.433571 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b421fc-ca06-40d2-bef1-d598d4ff5ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:03:09.435249 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-15b421fc-ca06-40d2-bef1-d598d4ff5ba3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:09.444416 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b421fc-ca06-40d2-bef1-d598d4ff5ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:03:09.445821 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b421fc-ca06-40d2-bef1-d598d4ff5ba3 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:03:09.454147 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1475/2935] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:03:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:03:10.085316 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31c807-a4d5-462b-8f08-8dc1179ba4e6 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:03:10.087123 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9b31c807-a4d5-462b-8f08-8dc1179ba4e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:10.103383 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b31c807-a4d5-462b-8f08-8dc1179ba4e6 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:03:10.109072 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57252bf-1269-4317-ba6e-7e775d85b27c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:03:10.109792 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57252bf-1269-4317-ba6e-7e775d85b27c service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:03:10.110286 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57252bf-1269-4317-ba6e-7e775d85b27c service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:03:10.110516 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57252bf-1269-4317-ba6e-7e775d85b27c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:03:10.112248 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b57252bf-1269-4317-ba6e-7e775d85b27c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:10.124922 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57252bf-1269-4317-ba6e-7e775d85b27c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:03:10.126536 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57252bf-1269-4317-ba6e-7e775d85b27c service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:03:10.137968 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1461/2936] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:03:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:03:15.274164 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b421fc-ca06-40d2-bef1-d598d4ff5ba3 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:03:15.287708 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b421fc-ca06-40d2-bef1-d598d4ff5ba3 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:03:15.293727 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d18f3c-9587-4c81-bd2b-01ed11f73a8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:03:15.294424 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d18f3c-9587-4c81-bd2b-01ed11f73a8f service nova] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:03:15.294766 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d18f3c-9587-4c81-bd2b-01ed11f73a8f service nova] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:03:15.295107 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d18f3c-9587-4c81-bd2b-01ed11f73a8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:03:15.296872 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-76d18f3c-9587-4c81-bd2b-01ed11f73a8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:03:15.309717 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d18f3c-9587-4c81-bd2b-01ed11f73a8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:03:15.311328 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d18f3c-9587-4c81-bd2b-01ed11f73a8f service nova] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:03:15.321707 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1476/2937] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:03:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:03:20.132807 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=48 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:03:25.317197 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=93 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:04:04.335156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57252bf-1269-4317-ba6e-7e775d85b27c service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:04.345162 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57252bf-1269-4317-ba6e-7e775d85b27c service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:04.348637 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc7613-91a2-42f5-8fd9-33232c5836df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:04.349220 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc7613-91a2-42f5-8fd9-33232c5836df service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:04.349494 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc7613-91a2-42f5-8fd9-33232c5836df service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:04.349761 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc7613-91a2-42f5-8fd9-33232c5836df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:04.351460 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-50dc7613-91a2-42f5-8fd9-33232c5836df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:04.412291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc7613-91a2-42f5-8fd9-33232c5836df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:04.413691 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc7613-91a2-42f5-8fd9-33232c5836df service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:04.423554 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1462/2938] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:04:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:04:04.862892 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d18f3c-9587-4c81-bd2b-01ed11f73a8f service nova] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:04.864464 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-76d18f3c-9587-4c81-bd2b-01ed11f73a8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:04.879333 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d18f3c-9587-4c81-bd2b-01ed11f73a8f service nova] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:04.884551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31558726-fd1b-4d19-96c5-573b73ce07e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:04.885156 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31558726-fd1b-4d19-96c5-573b73ce07e2 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:04.885440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31558726-fd1b-4d19-96c5-573b73ce07e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:04.885721 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31558726-fd1b-4d19-96c5-573b73ce07e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:04.887996 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-31558726-fd1b-4d19-96c5-573b73ce07e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:04.899671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31558726-fd1b-4d19-96c5-573b73ce07e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:04.901383 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31558726-fd1b-4d19-96c5-573b73ce07e2 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:04.914238 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1477/2939] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:04:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:04:04.927742 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc7613-91a2-42f5-8fd9-33232c5836df service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:04.929399 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-50dc7613-91a2-42f5-8fd9-33232c5836df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:04.942172 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc7613-91a2-42f5-8fd9-33232c5836df service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:04.948364 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcd68b7-e61a-4a5f-be79-84cc58a2012d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:04.948999 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd68b7-e61a-4a5f-be79-84cc58a2012d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:04.949478 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd68b7-e61a-4a5f-be79-84cc58a2012d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:04.949589 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcd68b7-e61a-4a5f-be79-84cc58a2012d admin admin] PATH_INFO: `/v3/users/10f2730695454358bc1707dd0815d880` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:04.951213 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd68b7-e61a-4a5f-be79-84cc58a2012d admin admin] RBAC: Authorizing `identity:delete_user(user_id=10f2730695454358bc1707dd0815d880)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:04.953137 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcd68b7-e61a-4a5f-be79-84cc58a2012d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:05.001183 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2fcd68b7-e61a-4a5f-be79-84cc58a2012d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10f2730695454358bc1707dd0815d880'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:04:05.008282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2fcd68b7-e61a-4a5f-be79-84cc58a2012d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10f2730695454358bc1707dd0815d880'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:04:05.020753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2fcd68b7-e61a-4a5f-be79-84cc58a2012d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10f2730695454358bc1707dd0815d880'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:04:05.021380 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2fcd68b7-e61a-4a5f-be79-84cc58a2012d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10f2730695454358bc1707dd0815d880'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:04:05.028432 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2fcd68b7-e61a-4a5f-be79-84cc58a2012d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10f2730695454358bc1707dd0815d880'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:04:05.034254 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1463/2940] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 17:04:04 2022] DELETE /identity/v3/users/10f2730695454358bc1707dd0815d880 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 17:04:05.208622 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31558726-fd1b-4d19-96c5-573b73ce07e2 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:05.219329 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31558726-fd1b-4d19-96c5-573b73ce07e2 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:05.280636 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552f13b-dd07-47ba-b39a-6a58416c5dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:05.281271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552f13b-dd07-47ba-b39a-6a58416c5dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:05.281555 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552f13b-dd07-47ba-b39a-6a58416c5dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:05.281833 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552f13b-dd07-47ba-b39a-6a58416c5dfc admin admin] PATH_INFO: `/v3/projects/5eb1e42761a74d7eb8509ac0f4d1c1db` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:05.282985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552f13b-dd07-47ba-b39a-6a58416c5dfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eb1e42761a74d7eb8509ac0f4d1c1db)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:05.284885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552f13b-dd07-47ba-b39a-6a58416c5dfc admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:05.331775 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2552f13b-dd07-47ba-b39a-6a58416c5dfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eb1e42761a74d7eb8509ac0f4d1c1db', 'request_id': 'req-2552f13b-dd07-47ba-b39a-6a58416c5dfc'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:04:05.339065 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2552f13b-dd07-47ba-b39a-6a58416c5dfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eb1e42761a74d7eb8509ac0f4d1c1db', 'request_id': 'req-2552f13b-dd07-47ba-b39a-6a58416c5dfc'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:04:05.345202 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2552f13b-dd07-47ba-b39a-6a58416c5dfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eb1e42761a74d7eb8509ac0f4d1c1db', 'request_id': 'req-2552f13b-dd07-47ba-b39a-6a58416c5dfc'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:04:05.355816 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2552f13b-dd07-47ba-b39a-6a58416c5dfc admin admin] The token cache is being invalidate because project 5eb1e42761a74d7eb8509ac0f4d1c1db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 17:04:05.356108 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2552f13b-dd07-47ba-b39a-6a58416c5dfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:04:05.357259 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-2552f13b-dd07-47ba-b39a-6a58416c5dfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:04:05.358742 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1478/2941] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 17:04:05 2022] DELETE /identity/v3/projects/5eb1e42761a74d7eb8509ac0f4d1c1db => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 17:04:05.364885 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10a1c4-63a6-490f-8562-baad350302ec None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:05.365332 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10a1c4-63a6-490f-8562-baad350302ec None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:05.365419 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10a1c4-63a6-490f-8562-baad350302ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:05.412521 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fc10a1c4-63a6-490f-8562-baad350302ec None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 17:04:05.413715 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fc10a1c4-63a6-490f-8562-baad350302ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:05.451093 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:04:05.473891 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1464/2942] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 17:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 17:04:05.482495 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4b522-d2dd-476a-b9af-bf8bcf23d1f6 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:05.482785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4b522-d2dd-476a-b9af-bf8bcf23d1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:05.483066 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4b522-d2dd-476a-b9af-bf8bcf23d1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:05.503286 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-e3a4b522-d2dd-476a-b9af-bf8bcf23d1f6 None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 17:04:05.504492 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-e3a4b522-d2dd-476a-b9af-bf8bcf23d1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:05.512664 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1479/2943] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 17:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 17:04:05.516963 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10a1c4-63a6-490f-8562-baad350302ec None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:05.526578 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10a1c4-63a6-490f-8562-baad350302ec None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:05.531079 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeaff54-692b-40d9-9296-49f370433bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:05.531665 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaff54-692b-40d9-9296-49f370433bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:05.531944 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaff54-692b-40d9-9296-49f370433bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:05.532214 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeaff54-692b-40d9-9296-49f370433bc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:05.532854 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeaff54-692b-40d9-9296-49f370433bc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 21 17:04:05.533154 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeaff54-692b-40d9-9296-49f370433bc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:05.534833 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeaff54-692b-40d9-9296-49f370433bc0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:05.546688 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1465/2944] 10.222.0.64 () {58 vars in 1153 bytes} [Wed Sep 21 17:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 21 17:04:05.550882 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4b522-d2dd-476a-b9af-bf8bcf23d1f6 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:05.560649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4b522-d2dd-476a-b9af-bf8bcf23d1f6 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:05.565422 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09c0d4-3909-47b3-b4e6-ad6c8cb105a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:05.565997 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09c0d4-3909-47b3-b4e6-ad6c8cb105a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:05.566393 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09c0d4-3909-47b3-b4e6-ad6c8cb105a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:05.566673 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09c0d4-3909-47b3-b4e6-ad6c8cb105a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:05.567310 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09c0d4-3909-47b3-b4e6-ad6c8cb105a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:05.569196 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09c0d4-3909-47b3-b4e6-ad6c8cb105a1 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:05.606068 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1480/2945] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 17:04:05 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 17:04:05.610693 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeaff54-692b-40d9-9296-49f370433bc0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:05.620663 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeaff54-692b-40d9-9296-49f370433bc0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:05.676905 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f2a10-daea-4692-b345-c59dead2ceaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:05.677431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f2a10-daea-4692-b345-c59dead2ceaa admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:05.677712 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f2a10-daea-4692-b345-c59dead2ceaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:05.677988 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f2a10-daea-4692-b345-c59dead2ceaa admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:05.678650 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f2a10-daea-4692-b345-c59dead2ceaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:05.680579 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f2a10-daea-4692-b345-c59dead2ceaa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:05.702474 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1466/2946] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 17:04:05 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 21 17:04:05.706378 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09c0d4-3909-47b3-b4e6-ad6c8cb105a1 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:05.716803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09c0d4-3909-47b3-b4e6-ad6c8cb105a1 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:05.721354 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4dc07e-1923-4e7c-964d-ae19753d8df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:05.721942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dc07e-1923-4e7c-964d-ae19753d8df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:05.722257 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dc07e-1923-4e7c-964d-ae19753d8df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:05.722533 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dc07e-1923-4e7c-964d-ae19753d8df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:05.723172 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4dc07e-1923-4e7c-964d-ae19753d8df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:05.724741 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4dc07e-1923-4e7c-964d-ae19753d8df2 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:05.734991 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1481/2947] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 17:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 17:04:05.738641 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f2a10-daea-4692-b345-c59dead2ceaa admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:05.749387 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f2a10-daea-4692-b345-c59dead2ceaa admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:05.753934 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d95f2a-bba0-4960-8de6-029becd6c8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:05.754553 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d95f2a-bba0-4960-8de6-029becd6c8ab admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:05.754831 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d95f2a-bba0-4960-8de6-029becd6c8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:05.755102 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d95f2a-bba0-4960-8de6-029becd6c8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:05.755731 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d95f2a-bba0-4960-8de6-029becd6c8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:05.757290 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d95f2a-bba0-4960-8de6-029becd6c8ab admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:05.767692 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1467/2948] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 17:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 17:04:05.771885 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4dc07e-1923-4e7c-964d-ae19753d8df2 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:05.782158 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4dc07e-1923-4e7c-964d-ae19753d8df2 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:05.786716 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881b442-fbcf-49dc-8bb2-c1f37453c767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:05.787357 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881b442-fbcf-49dc-8bb2-c1f37453c767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:05.787635 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881b442-fbcf-49dc-8bb2-c1f37453c767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:05.787906 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881b442-fbcf-49dc-8bb2-c1f37453c767 admin admin] PATH_INFO: `/v3/projects/67d9fbc339f64ce8b323195b51b06390/users/2a4d5e6077b642c4bc9bc3be27c15092/roles/20378c15bf524a4c960b49d2c53cf6b2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:05.802763 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881b442-fbcf-49dc-8bb2-c1f37453c767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d9fbc339f64ce8b323195b51b06390, user_id=2a4d5e6077b642c4bc9bc3be27c15092, role_id=20378c15bf524a4c960b49d2c53cf6b2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:05.805353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881b442-fbcf-49dc-8bb2-c1f37453c767 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:05.818671 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1482/2949] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 17:04:05 2022] PUT /identity/v3/projects/67d9fbc339f64ce8b323195b51b06390/users/2a4d5e6077b642c4bc9bc3be27c15092/roles/20378c15bf524a4c960b49d2c53cf6b2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 17:04:05.823577 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e948a4-bb99-4d44-8cf2-1f7a9bf9747d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:05.823873 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e948a4-bb99-4d44-8cf2-1f7a9bf9747d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:05.824174 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e948a4-bb99-4d44-8cf2-1f7a9bf9747d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:05.843757 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-46e948a4-bb99-4d44-8cf2-1f7a9bf9747d None None] MFA Rules not processed for user `b9a31b5d481f471a90714fa648bfed35`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 17:04:05.844943 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-46e948a4-bb99-4d44-8cf2-1f7a9bf9747d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:05.901366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1468/2950] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 17:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 17:04:05.911364 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881b442-fbcf-49dc-8bb2-c1f37453c767 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:05.913036 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-3881b442-fbcf-49dc-8bb2-c1f37453c767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:05.920186 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:04:05.971301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881b442-fbcf-49dc-8bb2-c1f37453c767 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:05.974815 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae845b4-6e27-47b5-bde1-27537863eea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:05.975412 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae845b4-6e27-47b5-bde1-27537863eea5 service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:05.975689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae845b4-6e27-47b5-bde1-27537863eea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:05.975959 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae845b4-6e27-47b5-bde1-27537863eea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:05.985228 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae845b4-6e27-47b5-bde1-27537863eea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:05.986674 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae845b4-6e27-47b5-bde1-27537863eea5 service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:05.995831 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1483/2951] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:04:10.218635 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063a07e-1b75-44cf-b820-64722998e326 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:10.219126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063a07e-1b75-44cf-b820-64722998e326 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:10.219126 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8063a07e-1b75-44cf-b820-64722998e326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:10.251452 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8063a07e-1b75-44cf-b820-64722998e326 None None] MFA Rules not processed for user `2a4d5e6077b642c4bc9bc3be27c15092`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 17:04:10.252893 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8063a07e-1b75-44cf-b820-64722998e326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:10.294957 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:04:10.322600 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1469/2952] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 17:04:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 17:04:10.327301 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae845b4-6e27-47b5-bde1-27537863eea5 service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:10.337579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae845b4-6e27-47b5-bde1-27537863eea5 service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:10.343638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8717f45-71af-4a2f-86d3-0bee7417bdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:10.344248 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717f45-71af-4a2f-86d3-0bee7417bdea admin admin] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:10.344529 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717f45-71af-4a2f-86d3-0bee7417bdea admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:10.344803 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8717f45-71af-4a2f-86d3-0bee7417bdea admin admin] PATH_INFO: `/v3/projects` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:10.345440 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8717f45-71af-4a2f-86d3-0bee7417bdea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:10.347465 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8717f45-71af-4a2f-86d3-0bee7417bdea admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:10.378544 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1484/2953] 10.222.0.64 () {60 vars in 1153 bytes} [Wed Sep 21 17:04:10 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 17:04:10.382291 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063a07e-1b75-44cf-b820-64722998e326 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:10.393385 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8063a07e-1b75-44cf-b820-64722998e326 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:10.452141 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7681c-2f9d-4a48-b515-4ca6a7782298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:10.452766 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7681c-2f9d-4a48-b515-4ca6a7782298 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:10.453049 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7681c-2f9d-4a48-b515-4ca6a7782298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:10.453330 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7681c-2f9d-4a48-b515-4ca6a7782298 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:10.453980 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7681c-2f9d-4a48-b515-4ca6a7782298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:10.455925 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7681c-2f9d-4a48-b515-4ca6a7782298 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:10.479473 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1470/2954] 10.222.0.64 () {60 vars in 1144 bytes} [Wed Sep 21 17:04:10 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 21 17:04:10.487412 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8717f45-71af-4a2f-86d3-0bee7417bdea admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:10.500209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8717f45-71af-4a2f-86d3-0bee7417bdea admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:10.504868 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df78df7-bfa6-4aba-a595-8309ea2b9618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:10.505621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df78df7-bfa6-4aba-a595-8309ea2b9618 admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:10.505902 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df78df7-bfa6-4aba-a595-8309ea2b9618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:10.506209 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df78df7-bfa6-4aba-a595-8309ea2b9618 admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:10.506891 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df78df7-bfa6-4aba-a595-8309ea2b9618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:10.508494 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df78df7-bfa6-4aba-a595-8309ea2b9618 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:10.522790 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1485/2955] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 17:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 17:04:10.524050 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7681c-2f9d-4a48-b515-4ca6a7782298 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:10.535252 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7681c-2f9d-4a48-b515-4ca6a7782298 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:10.540074 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdcdcc-7f64-4c23-8f38-dfd38e33be59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:10.540765 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdcdcc-7f64-4c23-8f38-dfd38e33be59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:10.541042 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdcdcc-7f64-4c23-8f38-dfd38e33be59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:10.541315 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdcdcc-7f64-4c23-8f38-dfd38e33be59 admin admin] PATH_INFO: `/v3/projects/dc7a7b12efdc45f8bb0ae59df9b2428f/users/31571d28cc914c4ba33f82e7959bfef2/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:10.559070 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdcdcc-7f64-4c23-8f38-dfd38e33be59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7a7b12efdc45f8bb0ae59df9b2428f, user_id=31571d28cc914c4ba33f82e7959bfef2, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:10.562542 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdcdcc-7f64-4c23-8f38-dfd38e33be59 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:10.577218 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1471/2956] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 17:04:10 2022] PUT /identity/v3/projects/dc7a7b12efdc45f8bb0ae59df9b2428f/users/31571d28cc914c4ba33f82e7959bfef2/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 17:04:10.581347 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df78df7-bfa6-4aba-a595-8309ea2b9618 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:10.592649 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df78df7-bfa6-4aba-a595-8309ea2b9618 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:10.651808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c8b62-f6d1-4f28-ab9f-97a552ceabaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:10.652418 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c8b62-f6d1-4f28-ab9f-97a552ceabaf admin admin] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:10.652699 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c8b62-f6d1-4f28-ab9f-97a552ceabaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:10.653597 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c8b62-f6d1-4f28-ab9f-97a552ceabaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:10.654287 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c8b62-f6d1-4f28-ab9f-97a552ceabaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:10.655871 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c8b62-f6d1-4f28-ab9f-97a552ceabaf admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:10.665967 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1486/2957] 10.222.0.64 () {58 vars in 1122 bytes} [Wed Sep 21 17:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 21 17:04:10.670301 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdcdcc-7f64-4c23-8f38-dfd38e33be59 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:10.680681 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdcdcc-7f64-4c23-8f38-dfd38e33be59 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:10.685272 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c82cd-f34e-4eeb-95d1-a7633b3a1dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:10.685911 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c82cd-f34e-4eeb-95d1-a7633b3a1dad admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:10.686217 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c82cd-f34e-4eeb-95d1-a7633b3a1dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:10.686495 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c82cd-f34e-4eeb-95d1-a7633b3a1dad admin admin] PATH_INFO: `/v3/projects/dc7a7b12efdc45f8bb0ae59df9b2428f/users/31571d28cc914c4ba33f82e7959bfef2/roles/459d01a218314543a7a8329e88b78331` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:10.688431 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c82cd-f34e-4eeb-95d1-a7633b3a1dad admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7a7b12efdc45f8bb0ae59df9b2428f, user_id=31571d28cc914c4ba33f82e7959bfef2, role_id=459d01a218314543a7a8329e88b78331)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:10.691151 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c82cd-f34e-4eeb-95d1-a7633b3a1dad admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:10.703142 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1472/2958] 10.222.0.64 () {60 vars in 1483 bytes} [Wed Sep 21 17:04:10 2022] PUT /identity/v3/projects/dc7a7b12efdc45f8bb0ae59df9b2428f/users/31571d28cc914c4ba33f82e7959bfef2/roles/459d01a218314543a7a8329e88b78331 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 21 17:04:15.115731 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce7b9a-259a-40b2-9336-b2bbf19fced4 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:15.116918 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce7b9a-259a-40b2-9336-b2bbf19fced4 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:15.117656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce7b9a-259a-40b2-9336-b2bbf19fced4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:15.165956 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-18ce7b9a-259a-40b2-9336-b2bbf19fced4 None None] MFA Rules not processed for user `31571d28cc914c4ba33f82e7959bfef2`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 17:04:15.167222 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-18ce7b9a-259a-40b2-9336-b2bbf19fced4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:15.247588 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:04:15.261550 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1487/2959] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 17:04:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 17:04:15.278810 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c82cd-f34e-4eeb-95d1-a7633b3a1dad admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:15.281367 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fc8c82cd-f34e-4eeb-95d1-a7633b3a1dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:15.370316 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c82cd-f34e-4eeb-95d1-a7633b3a1dad admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:15.380224 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b898a-f7fd-4cb7-86bd-a65526d0c003 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:15.380857 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b898a-f7fd-4cb7-86bd-a65526d0c003 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:15.381140 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b898a-f7fd-4cb7-86bd-a65526d0c003 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:15.381415 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b898a-f7fd-4cb7-86bd-a65526d0c003 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:15.411058 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:04:15.448016 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b898a-f7fd-4cb7-86bd-a65526d0c003 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:15.449598 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b898a-f7fd-4cb7-86bd-a65526d0c003 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:15.459282 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1473/2960] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:04:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:04:16.685342 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099d486-13ed-4eec-8ca8-32bf8eed4822 None None] REQUEST_METHOD: `POST` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:16.685842 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099d486-13ed-4eec-8ca8-32bf8eed4822 None None] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:16.685934 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099d486-13ed-4eec-8ca8-32bf8eed4822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:16.705401 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.auth.core [None req-9099d486-13ed-4eec-8ca8-32bf8eed4822 None None] MFA Rules not processed for user `2a4d5e6077b642c4bc9bc3be27c15092`. Rule list: `[]` (Enabled: `True`). {{(pid=81426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 21 17:04:16.706647 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9099d486-13ed-4eec-8ca8-32bf8eed4822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:16.713751 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1488/2961] 10.222.0.64 () {58 vars in 958 bytes} [Wed Sep 21 17:04:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 21 17:04:25.455341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=15 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:04:26.699506 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=5 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:04:32.230266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b898a-f7fd-4cb7-86bd-a65526d0c003 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:32.296662 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b898a-f7fd-4cb7-86bd-a65526d0c003 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:32.300518 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5125763-5cd2-49c8-a30d-dd76c1dd576a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:32.301152 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5125763-5cd2-49c8-a30d-dd76c1dd576a service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:32.301429 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5125763-5cd2-49c8-a30d-dd76c1dd576a service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:32.301725 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5125763-5cd2-49c8-a30d-dd76c1dd576a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:32.303477 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f5125763-5cd2-49c8-a30d-dd76c1dd576a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:32.326004 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5125763-5cd2-49c8-a30d-dd76c1dd576a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:32.327476 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5125763-5cd2-49c8-a30d-dd76c1dd576a service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:32.335490 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1474/2962] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:04:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:04:42.331883 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=35 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:04:48.191298 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099d486-13ed-4eec-8ca8-32bf8eed4822 None None] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:48.192738 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-9099d486-13ed-4eec-8ca8-32bf8eed4822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:48.205849 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9099d486-13ed-4eec-8ca8-32bf8eed4822 None None] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:48.209353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2d9af-4a28-4521-908b-bf62f41504bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:48.209942 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2d9af-4a28-4521-908b-bf62f41504bb service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:48.210271 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2d9af-4a28-4521-908b-bf62f41504bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:48.210545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb2d9af-4a28-4521-908b-bf62f41504bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:48.220540 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2d9af-4a28-4521-908b-bf62f41504bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:48.222008 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb2d9af-4a28-4521-908b-bf62f41504bb service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:48.231182 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1489/2963] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:04:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:04:49.778497 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5125763-5cd2-49c8-a30d-dd76c1dd576a service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:04:49.780094 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f5125763-5cd2-49c8-a30d-dd76c1dd576a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:49.796873 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5125763-5cd2-49c8-a30d-dd76c1dd576a service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:04:49.802394 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82f021-9b8f-453b-ab44-ae681732bc29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:04:49.803009 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82f021-9b8f-453b-ab44-ae681732bc29 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:04:49.803288 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82f021-9b8f-453b-ab44-ae681732bc29 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:04:49.803560 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82f021-9b8f-453b-ab44-ae681732bc29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:04:49.805302 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1e82f021-9b8f-453b-ab44-ae681732bc29 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:04:49.818768 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82f021-9b8f-453b-ab44-ae681732bc29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:04:49.820360 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82f021-9b8f-453b-ab44-ae681732bc29 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:04:49.831853 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1475/2964] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:04:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:04:58.227146 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=6 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:04:59.827449 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:06:43.928246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2d9af-4a28-4521-908b-bf62f41504bb service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:06:43.929725 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-efb2d9af-4a28-4521-908b-bf62f41504bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:06:44.013616 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb2d9af-4a28-4521-908b-bf62f41504bb service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:06:44.019214 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecf0c1-559f-4ae6-904b-a5335742ed23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:06:44.019831 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecf0c1-559f-4ae6-904b-a5335742ed23 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:06:44.020115 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecf0c1-559f-4ae6-904b-a5335742ed23 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:06:44.020390 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecf0c1-559f-4ae6-904b-a5335742ed23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:06:44.022188 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-fbecf0c1-559f-4ae6-904b-a5335742ed23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:06:44.035754 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecf0c1-559f-4ae6-904b-a5335742ed23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:06:44.037388 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecf0c1-559f-4ae6-904b-a5335742ed23 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:06:44.048993 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1490/2965] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:06:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 17:06:54.044033 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=37 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:06:57.377156 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82f021-9b8f-453b-ab44-ae681732bc29 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:06:57.387341 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82f021-9b8f-453b-ab44-ae681732bc29 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:06:57.391236 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bb0ea-f726-4045-ac0a-3721feaf84bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:06:57.391821 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bb0ea-f726-4045-ac0a-3721feaf84bf service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:06:57.392114 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bb0ea-f726-4045-ac0a-3721feaf84bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:06:57.392366 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2bb0ea-f726-4045-ac0a-3721feaf84bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:06:57.394180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3c2bb0ea-f726-4045-ac0a-3721feaf84bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:06:57.403990 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bb0ea-f726-4045-ac0a-3721feaf84bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:06:57.405537 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2bb0ea-f726-4045-ac0a-3721feaf84bf service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:06:57.414624 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1476/2966] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:06:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:06:57.501017 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecf0c1-559f-4ae6-904b-a5335742ed23 service placement] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:06:57.510579 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecf0c1-559f-4ae6-904b-a5335742ed23 service placement] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:06:57.514210 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317b767-07cd-4719-bdb2-5c4d7bad382e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:06:57.514808 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317b767-07cd-4719-bdb2-5c4d7bad382e service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:06:57.515084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317b767-07cd-4719-bdb2-5c4d7bad382e service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:06:57.515353 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317b767-07cd-4719-bdb2-5c4d7bad382e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:06:57.517023 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-5317b767-07cd-4719-bdb2-5c4d7bad382e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:06:57.530355 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317b767-07cd-4719-bdb2-5c4d7bad382e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:06:57.531746 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317b767-07cd-4719-bdb2-5c4d7bad382e service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:06:57.542776 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1491/2967] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:06:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:07:07.410753 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:07:07.538089 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=6 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:08:23.556281 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bb0ea-f726-4045-ac0a-3721feaf84bf service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:08:23.570534 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2bb0ea-f726-4045-ac0a-3721feaf84bf service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:08:23.576307 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56109435-8be3-46c7-ad32-27e327900f68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0acf23d224e4b16ad03cdd9bf7cae8e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'service', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:08:23.576931 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56109435-8be3-46c7-ad32-27e327900f68 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:08:23.577210 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56109435-8be3-46c7-ad32-27e327900f68 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:08:23.577480 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56109435-8be3-46c7-ad32-27e327900f68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:08:23.579541 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-56109435-8be3-46c7-ad32-27e327900f68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:08:23.597199 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56109435-8be3-46c7-ad32-27e327900f68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:08:23.598977 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56109435-8be3-46c7-ad32-27e327900f68 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:08:23.611686 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1477/2968] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:08:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:08:23.682883 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317b767-07cd-4719-bdb2-5c4d7bad382e service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:08:23.701785 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317b767-07cd-4719-bdb2-5c4d7bad382e service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:08:23.706231 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c3a73-155b-43cf-8272-8f913db6aa4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:08:23.706932 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c3a73-155b-43cf-8272-8f913db6aa4d service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:08:23.707453 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c3a73-155b-43cf-8272-8f913db6aa4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:08:23.707796 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260c3a73-155b-43cf-8272-8f913db6aa4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:08:23.709876 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-260c3a73-155b-43cf-8272-8f913db6aa4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:08:23.725737 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c3a73-155b-43cf-8272-8f913db6aa4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:08:23.727307 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260c3a73-155b-43cf-8272-8f913db6aa4d service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:08:23.737894 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1492/2969] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:08:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:08:23.743287 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56109435-8be3-46c7-ad32-27e327900f68 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:08:23.754580 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56109435-8be3-46c7-ad32-27e327900f68 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:08:23.758972 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d51954-066b-4e89-a32f-c9ca2c54986c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:08:23.759651 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d51954-066b-4e89-a32f-c9ca2c54986c service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:08:23.760000 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d51954-066b-4e89-a32f-c9ca2c54986c service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:08:23.760343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d51954-066b-4e89-a32f-c9ca2c54986c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:08:23.762190 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e7d51954-066b-4e89-a32f-c9ca2c54986c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:08:23.779045 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d51954-066b-4e89-a32f-c9ca2c54986c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:08:23.780427 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d51954-066b-4e89-a32f-c9ca2c54986c service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:08:23.792655 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1478/2970] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:08:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 21 17:08:24.981098 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c3a73-155b-43cf-8272-8f913db6aa4d service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:08:24.990740 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260c3a73-155b-43cf-8272-8f913db6aa4d service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:08:24.994382 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25a108-f1b2-468d-833a-90072e99e44c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e175b8de037d45d48cbddd64e552f6f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:08:24.994982 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25a108-f1b2-468d-833a-90072e99e44c service neutron] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:08:24.995262 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25a108-f1b2-468d-833a-90072e99e44c service neutron] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:08:24.995536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25a108-f1b2-468d-833a-90072e99e44c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:08:24.997246 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-be25a108-f1b2-468d-833a-90072e99e44c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:08:25.010621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25a108-f1b2-468d-833a-90072e99e44c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:08:25.012042 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25a108-f1b2-468d-833a-90072e99e44c service neutron] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:08:25.022896 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1493/2971] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:08:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:08:30.550357 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d51954-066b-4e89-a32f-c9ca2c54986c service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:08:30.552343 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e7d51954-066b-4e89-a32f-c9ca2c54986c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:08:30.569327 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d51954-066b-4e89-a32f-c9ca2c54986c service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:08:30.575088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79843483-6cd4-4b7e-bee7-6d1e3c77ed96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:08:30.575710 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79843483-6cd4-4b7e-bee7-6d1e3c77ed96 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:08:30.575991 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79843483-6cd4-4b7e-bee7-6d1e3c77ed96 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:08:30.576266 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79843483-6cd4-4b7e-bee7-6d1e3c77ed96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:08:30.579371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-79843483-6cd4-4b7e-bee7-6d1e3c77ed96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:08:30.593180 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79843483-6cd4-4b7e-bee7-6d1e3c77ed96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:08:30.595410 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79843483-6cd4-4b7e-bee7-6d1e3c77ed96 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:08:30.609928 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1479/2972] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:08:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:08:33.700202 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25a108-f1b2-468d-833a-90072e99e44c service neutron] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:08:33.788521 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25a108-f1b2-468d-833a-90072e99e44c service neutron] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:08:33.793002 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:08:33.793621 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:08:33.793897 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:08:33.794202 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] PATH_INFO: `/v3/users/2a4d5e6077b642c4bc9bc3be27c15092` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:08:33.795300 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a4d5e6077b642c4bc9bc3be27c15092)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:08:33.797176 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:08:33.843190 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a4d5e6077b642c4bc9bc3be27c15092'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:33.851096 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a4d5e6077b642c4bc9bc3be27c15092'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:33.851801 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a4d5e6077b642c4bc9bc3be27c15092'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:33.860611 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a4d5e6077b642c4bc9bc3be27c15092'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:33.872656 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a4d5e6077b642c4bc9bc3be27c15092'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:33.878862 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1494/2973] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 17:08:33 2022] DELETE /identity/v3/users/2a4d5e6077b642c4bc9bc3be27c15092 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 17:08:34.051484 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79843483-6cd4-4b7e-bee7-6d1e3c77ed96 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:08:34.072413 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79843483-6cd4-4b7e-bee7-6d1e3c77ed96 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:08:34.133147 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:08:34.133802 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:08:34.134106 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:08:34.134382 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] PATH_INFO: `/v3/projects/67d9fbc339f64ce8b323195b51b06390` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:08:34.135485 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67d9fbc339f64ce8b323195b51b06390)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:08:34.137371 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:08:34.184734 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67d9fbc339f64ce8b323195b51b06390', 'request_id': 'req-52af0a9a-016f-4d03-ab6c-665a37c5b826'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.192653 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67d9fbc339f64ce8b323195b51b06390', 'request_id': 'req-52af0a9a-016f-4d03-ab6c-665a37c5b826'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.198491 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67d9fbc339f64ce8b323195b51b06390', 'request_id': 'req-52af0a9a-016f-4d03-ab6c-665a37c5b826'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.222034 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] The token cache is being invalidate because project 67d9fbc339f64ce8b323195b51b06390 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 17:08:34.222347 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.222956 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.224414 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1480/2974] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 17:08:34 2022] DELETE /identity/v3/projects/67d9fbc339f64ce8b323195b51b06390 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 17:08:34.228410 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:08:34.229836 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:08:34.242638 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6a983-31bc-4410-9a55-de32e9894660 admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:08:34.275689 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:08:34.300921 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:08:34.301551 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:08:34.301840 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:08:34.302134 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] PATH_INFO: `/v3/users/31571d28cc914c4ba33f82e7959bfef2` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:08:34.303254 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] RBAC: Authorizing `identity:delete_user(user_id=31571d28cc914c4ba33f82e7959bfef2)` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:08:34.305149 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:08:34.349329 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31571d28cc914c4ba33f82e7959bfef2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.356536 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31571d28cc914c4ba33f82e7959bfef2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.357159 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31571d28cc914c4ba33f82e7959bfef2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.364545 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31571d28cc914c4ba33f82e7959bfef2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.376695 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.notifications [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31571d28cc914c4ba33f82e7959bfef2'} {{(pid=81426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.382389 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1495/2975] 10.222.0.64 () {58 vars in 1224 bytes} [Wed Sep 21 17:08:34 2022] DELETE /identity/v3/users/31571d28cc914c4ba33f82e7959bfef2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 17:08:34.554702 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:08:34.556144 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:08:34.570088 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af0a9a-016f-4d03-ab6c-665a37c5b826 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:08:34.609824 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:08:34.629687 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0595209e-9d48-4255-849a-37b29d4cddc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9a31b5d481f471a90714fa648bfed35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '92877a675fac490b8473d8157aef4217', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:08:34.630321 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595209e-9d48-4255-849a-37b29d4cddc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:08:34.630602 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595209e-9d48-4255-849a-37b29d4cddc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:08:34.630879 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0595209e-9d48-4255-849a-37b29d4cddc4 admin admin] PATH_INFO: `/v3/projects/dc7a7b12efdc45f8bb0ae59df9b2428f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:08:34.632017 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595209e-9d48-4255-849a-37b29d4cddc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc7a7b12efdc45f8bb0ae59df9b2428f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:08:34.634001 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0595209e-9d48-4255-849a-37b29d4cddc4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:08:34.681487 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0595209e-9d48-4255-849a-37b29d4cddc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc7a7b12efdc45f8bb0ae59df9b2428f', 'request_id': 'req-0595209e-9d48-4255-849a-37b29d4cddc4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.689411 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0595209e-9d48-4255-849a-37b29d4cddc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc7a7b12efdc45f8bb0ae59df9b2428f', 'request_id': 'req-0595209e-9d48-4255-849a-37b29d4cddc4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.695237 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0595209e-9d48-4255-849a-37b29d4cddc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc7a7b12efdc45f8bb0ae59df9b2428f', 'request_id': 'req-0595209e-9d48-4255-849a-37b29d4cddc4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.706358 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0595209e-9d48-4255-849a-37b29d4cddc4 admin admin] The token cache is being invalidate because project dc7a7b12efdc45f8bb0ae59df9b2428f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 21 17:08:34.706644 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0595209e-9d48-4255-849a-37b29d4cddc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.707280 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0595209e-9d48-4255-849a-37b29d4cddc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 21 17:08:34.708738 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1481/2976] 10.222.0.64 () {58 vars in 1233 bytes} [Wed Sep 21 17:08:34 2022] DELETE /identity/v3/projects/dc7a7b12efdc45f8bb0ae59df9b2428f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 21 17:08:44.379084 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=37,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:08:44.703386 dv-cinder-858621-1-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=33,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 21 17:11:48.931227 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] Authenticating user token {{(pid=81426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 21 17:11:48.932625 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:11:49.028912 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c4c1c-65c4-4c82-9352-f67f6db8489c admin admin] Validating token access rules against request {{(pid=81426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 21 17:11:49.086153 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9a846-0f7c-43cb-ab46-850a8aa19c59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55642294e8ff4f779510f4646e0e30a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c3c9e1d9331b4d4dbd3dd447937c42ad', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 21 17:11:49.086758 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9a846-0f7c-43cb-ab46-850a8aa19c59 service placement] REQUEST_METHOD: `GET` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 21 17:11:49.087038 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9a846-0f7c-43cb-ab46-850a8aa19c59 service placement] SCRIPT_NAME: `/identity` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 21 17:11:49.087313 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9a846-0f7c-43cb-ab46-850a8aa19c59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 21 17:11:49.089010 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.fernet_utils [None req-6cd9a846-0f7c-43cb-ab46-850a8aa19c59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 21 17:11:49.100436 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9a846-0f7c-43cb-ab46-850a8aa19c59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 21 17:11:49.101985 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9a846-0f7c-43cb-ab46-850a8aa19c59 service placement] RBAC: Authorization granted {{(pid=81426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 21 17:11:49.112260 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: [pid: 81426|app: 0|req: 1496/2977] 10.222.0.64 () {60 vars in 1365 bytes} [Wed Sep 21 17:11:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 21 17:11:59.107457 dv-cinder-858621-1-iscsi devstack@keystone.service[81426]: DEBUG dbcounter [-] [81426] Writing DB stats keystone:SELECT=71 {{(pid=81426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}